thomwolf HF staff commited on
Commit
1252ede
·
verified ·
1 Parent(s): 5a7c330
Files changed (4) hide show
  1. dist/index.html +2 -2
  2. dist/style.css +8 -0
  3. src/index.html +2 -2
  4. src/style.css +8 -0
dist/index.html CHANGED
@@ -297,10 +297,10 @@
297
 
298
  <p>Using this snippet [TODO: link to appendix A5], we can understand how memory is allocated throughout training. We can see that memory utilization is not a static thing but varies a lot during training and during a training step:</p>
299
 
300
- <div class="svg-container l-body-outset" id="svg-first_steps_memory_profile"> </div>
301
  <div class="info" id="svg-first_steps_memory_profile-info">Hover over the elements to see their details</div>
302
  <script src="../assets/images/first_steps_memory_profile.js"></script>
303
-
304
  <iframe id="plotFrame" src="assets/data/benchmarks/memory-profile.html" height="520" width="1000" scrolling="no" frameborder="0"></iframe>
305
 
306
  <p>Clearly the first step looks very different from the subsequent ones, but let’s first have a look at the general anatomy of a step: first the activations increase quickly as we do the forward pass, then during the backward pass the gradients build up and as the backward pass propagates, the stored activations used to compute the gradients are progressively cleared. Finally, we perform the optimization step during which we need all the gradients and then update the optimizer states before we start the next forward pass. </p>
 
297
 
298
  <p>Using this snippet [TODO: link to appendix A5], we can understand how memory is allocated throughout training. We can see that memory utilization is not a static thing but varies a lot during training and during a training step:</p>
299
 
300
+ <!-- <div class="svg-container l-body-outset" id="svg-first_steps_memory_profile"> </div>
301
  <div class="info" id="svg-first_steps_memory_profile-info">Hover over the elements to see their details</div>
302
  <script src="../assets/images/first_steps_memory_profile.js"></script>
303
+ -->
304
  <iframe id="plotFrame" src="assets/data/benchmarks/memory-profile.html" height="520" width="1000" scrolling="no" frameborder="0"></iframe>
305
 
306
  <p>Clearly the first step looks very different from the subsequent ones, but let’s first have a look at the general anatomy of a step: first the activations increase quickly as we do the forward pass, then during the backward pass the gradients build up and as the backward pass propagates, the stored activations used to compute the gradients are progressively cleared. Finally, we perform the optimization step during which we need all the gradients and then update the optimizer states before we start the next forward pass. </p>
dist/style.css CHANGED
@@ -159,6 +159,10 @@ d-contents > nav a.active {
159
  border-bottom-width: 1px;
160
  border-bottom-style: solid;
161
  border-bottom-color: rgba(0, 0, 0, 0.1);
 
 
 
 
162
  }
163
  }
164
 
@@ -189,6 +193,10 @@ d-contents a:hover {
189
  position: -webkit-sticky; /* For Safari */
190
  position: sticky;
191
  top: 10px; /* Adjust this value if needed */
 
 
 
 
192
  }
193
  }
194
 
 
159
  border-bottom-width: 1px;
160
  border-bottom-style: solid;
161
  border-bottom-color: rgba(0, 0, 0, 0.1);
162
+ overflow-y: scroll;
163
+ max-height: 75%;
164
+ scrollbar-width: none;
165
+ z-index: -100;
166
  }
167
  }
168
 
 
193
  position: -webkit-sticky; /* For Safari */
194
  position: sticky;
195
  top: 10px; /* Adjust this value if needed */
196
+ overflow-y: scroll;
197
+ max-height: 75%;
198
+ scrollbar-width: none;
199
+ z-index: -100;
200
  }
201
  }
202
 
src/index.html CHANGED
@@ -297,10 +297,10 @@
297
 
298
  <p>Using this snippet [TODO: link to appendix A5], we can understand how memory is allocated throughout training. We can see that memory utilization is not a static thing but varies a lot during training and during a training step:</p>
299
 
300
- <div class="svg-container l-body-outset" id="svg-first_steps_memory_profile"> </div>
301
  <div class="info" id="svg-first_steps_memory_profile-info">Hover over the elements to see their details</div>
302
  <script src="../assets/images/first_steps_memory_profile.js"></script>
303
-
304
  <iframe id="plotFrame" src="assets/data/benchmarks/memory-profile.html" height="520" width="1000" scrolling="no" frameborder="0"></iframe>
305
 
306
  <p>Clearly the first step looks very different from the subsequent ones, but let’s first have a look at the general anatomy of a step: first the activations increase quickly as we do the forward pass, then during the backward pass the gradients build up and as the backward pass propagates, the stored activations used to compute the gradients are progressively cleared. Finally, we perform the optimization step during which we need all the gradients and then update the optimizer states before we start the next forward pass. </p>
 
297
 
298
  <p>Using this snippet [TODO: link to appendix A5], we can understand how memory is allocated throughout training. We can see that memory utilization is not a static thing but varies a lot during training and during a training step:</p>
299
 
300
+ <!-- <div class="svg-container l-body-outset" id="svg-first_steps_memory_profile"> </div>
301
  <div class="info" id="svg-first_steps_memory_profile-info">Hover over the elements to see their details</div>
302
  <script src="../assets/images/first_steps_memory_profile.js"></script>
303
+ -->
304
  <iframe id="plotFrame" src="assets/data/benchmarks/memory-profile.html" height="520" width="1000" scrolling="no" frameborder="0"></iframe>
305
 
306
  <p>Clearly the first step looks very different from the subsequent ones, but let’s first have a look at the general anatomy of a step: first the activations increase quickly as we do the forward pass, then during the backward pass the gradients build up and as the backward pass propagates, the stored activations used to compute the gradients are progressively cleared. Finally, we perform the optimization step during which we need all the gradients and then update the optimizer states before we start the next forward pass. </p>
src/style.css CHANGED
@@ -159,6 +159,10 @@ d-contents > nav a.active {
159
  border-bottom-width: 1px;
160
  border-bottom-style: solid;
161
  border-bottom-color: rgba(0, 0, 0, 0.1);
 
 
 
 
162
  }
163
  }
164
 
@@ -189,6 +193,10 @@ d-contents a:hover {
189
  position: -webkit-sticky; /* For Safari */
190
  position: sticky;
191
  top: 10px; /* Adjust this value if needed */
 
 
 
 
192
  }
193
  }
194
 
 
159
  border-bottom-width: 1px;
160
  border-bottom-style: solid;
161
  border-bottom-color: rgba(0, 0, 0, 0.1);
162
+ overflow-y: scroll;
163
+ max-height: 75%;
164
+ scrollbar-width: none;
165
+ z-index: -100;
166
  }
167
  }
168
 
 
193
  position: -webkit-sticky; /* For Safari */
194
  position: sticky;
195
  top: 10px; /* Adjust this value if needed */
196
+ overflow-y: scroll;
197
+ max-height: 75%;
198
+ scrollbar-width: none;
199
+ z-index: -100;
200
  }
201
  }
202