Spaces:
Running
Running
update graphs (#14)
Browse files- update ToC (319dbdceef14374b1b5769357e0568f7acd74a88)
- dist/index.html +2 -2
- dist/style.css +8 -0
- src/index.html +2 -2
- src/style.css +8 -0
dist/index.html
CHANGED
@@ -297,10 +297,10 @@
|
|
297 |
|
298 |
<p>Using this snippet [TODO: link to appendix A5], we can understand how memory is allocated throughout training. We can see that memory utilization is not a static thing but varies a lot during training and during a training step:</p>
|
299 |
|
300 |
-
<div class="svg-container l-body-outset" id="svg-first_steps_memory_profile"> </div>
|
301 |
<div class="info" id="svg-first_steps_memory_profile-info">Hover over the elements to see their details</div>
|
302 |
<script src="../assets/images/first_steps_memory_profile.js"></script>
|
303 |
-
|
304 |
<iframe id="plotFrame" src="assets/data/benchmarks/memory-profile.html" height="520" width="1000" scrolling="no" frameborder="0"></iframe>
|
305 |
|
306 |
<p>Clearly the first step looks very different from the subsequent ones, but let’s first have a look at the general anatomy of a step: first the activations increase quickly as we do the forward pass, then during the backward pass the gradients build up and as the backward pass propagates, the stored activations used to compute the gradients are progressively cleared. Finally, we perform the optimization step during which we need all the gradients and then update the optimizer states before we start the next forward pass. </p>
|
|
|
297 |
|
298 |
<p>Using this snippet [TODO: link to appendix A5], we can understand how memory is allocated throughout training. We can see that memory utilization is not a static thing but varies a lot during training and during a training step:</p>
|
299 |
|
300 |
+
<!-- <div class="svg-container l-body-outset" id="svg-first_steps_memory_profile"> </div>
|
301 |
<div class="info" id="svg-first_steps_memory_profile-info">Hover over the elements to see their details</div>
|
302 |
<script src="../assets/images/first_steps_memory_profile.js"></script>
|
303 |
+
-->
|
304 |
<iframe id="plotFrame" src="assets/data/benchmarks/memory-profile.html" height="520" width="1000" scrolling="no" frameborder="0"></iframe>
|
305 |
|
306 |
<p>Clearly the first step looks very different from the subsequent ones, but let’s first have a look at the general anatomy of a step: first the activations increase quickly as we do the forward pass, then during the backward pass the gradients build up and as the backward pass propagates, the stored activations used to compute the gradients are progressively cleared. Finally, we perform the optimization step during which we need all the gradients and then update the optimizer states before we start the next forward pass. </p>
|
dist/style.css
CHANGED
@@ -159,6 +159,10 @@ d-contents > nav a.active {
|
|
159 |
border-bottom-width: 1px;
|
160 |
border-bottom-style: solid;
|
161 |
border-bottom-color: rgba(0, 0, 0, 0.1);
|
|
|
|
|
|
|
|
|
162 |
}
|
163 |
}
|
164 |
|
@@ -189,6 +193,10 @@ d-contents a:hover {
|
|
189 |
position: -webkit-sticky; /* For Safari */
|
190 |
position: sticky;
|
191 |
top: 10px; /* Adjust this value if needed */
|
|
|
|
|
|
|
|
|
192 |
}
|
193 |
}
|
194 |
|
|
|
159 |
border-bottom-width: 1px;
|
160 |
border-bottom-style: solid;
|
161 |
border-bottom-color: rgba(0, 0, 0, 0.1);
|
162 |
+
overflow-y: scroll;
|
163 |
+
max-height: 75%;
|
164 |
+
scrollbar-width: none;
|
165 |
+
z-index: -100;
|
166 |
}
|
167 |
}
|
168 |
|
|
|
193 |
position: -webkit-sticky; /* For Safari */
|
194 |
position: sticky;
|
195 |
top: 10px; /* Adjust this value if needed */
|
196 |
+
overflow-y: scroll;
|
197 |
+
max-height: 75%;
|
198 |
+
scrollbar-width: none;
|
199 |
+
z-index: -100;
|
200 |
}
|
201 |
}
|
202 |
|
src/index.html
CHANGED
@@ -297,10 +297,10 @@
|
|
297 |
|
298 |
<p>Using this snippet [TODO: link to appendix A5], we can understand how memory is allocated throughout training. We can see that memory utilization is not a static thing but varies a lot during training and during a training step:</p>
|
299 |
|
300 |
-
<div class="svg-container l-body-outset" id="svg-first_steps_memory_profile"> </div>
|
301 |
<div class="info" id="svg-first_steps_memory_profile-info">Hover over the elements to see their details</div>
|
302 |
<script src="../assets/images/first_steps_memory_profile.js"></script>
|
303 |
-
|
304 |
<iframe id="plotFrame" src="assets/data/benchmarks/memory-profile.html" height="520" width="1000" scrolling="no" frameborder="0"></iframe>
|
305 |
|
306 |
<p>Clearly the first step looks very different from the subsequent ones, but let’s first have a look at the general anatomy of a step: first the activations increase quickly as we do the forward pass, then during the backward pass the gradients build up and as the backward pass propagates, the stored activations used to compute the gradients are progressively cleared. Finally, we perform the optimization step during which we need all the gradients and then update the optimizer states before we start the next forward pass. </p>
|
|
|
297 |
|
298 |
<p>Using this snippet [TODO: link to appendix A5], we can understand how memory is allocated throughout training. We can see that memory utilization is not a static thing but varies a lot during training and during a training step:</p>
|
299 |
|
300 |
+
<!-- <div class="svg-container l-body-outset" id="svg-first_steps_memory_profile"> </div>
|
301 |
<div class="info" id="svg-first_steps_memory_profile-info">Hover over the elements to see their details</div>
|
302 |
<script src="../assets/images/first_steps_memory_profile.js"></script>
|
303 |
+
-->
|
304 |
<iframe id="plotFrame" src="assets/data/benchmarks/memory-profile.html" height="520" width="1000" scrolling="no" frameborder="0"></iframe>
|
305 |
|
306 |
<p>Clearly the first step looks very different from the subsequent ones, but let’s first have a look at the general anatomy of a step: first the activations increase quickly as we do the forward pass, then during the backward pass the gradients build up and as the backward pass propagates, the stored activations used to compute the gradients are progressively cleared. Finally, we perform the optimization step during which we need all the gradients and then update the optimizer states before we start the next forward pass. </p>
|
src/style.css
CHANGED
@@ -159,6 +159,10 @@ d-contents > nav a.active {
|
|
159 |
border-bottom-width: 1px;
|
160 |
border-bottom-style: solid;
|
161 |
border-bottom-color: rgba(0, 0, 0, 0.1);
|
|
|
|
|
|
|
|
|
162 |
}
|
163 |
}
|
164 |
|
@@ -189,6 +193,10 @@ d-contents a:hover {
|
|
189 |
position: -webkit-sticky; /* For Safari */
|
190 |
position: sticky;
|
191 |
top: 10px; /* Adjust this value if needed */
|
|
|
|
|
|
|
|
|
192 |
}
|
193 |
}
|
194 |
|
|
|
159 |
border-bottom-width: 1px;
|
160 |
border-bottom-style: solid;
|
161 |
border-bottom-color: rgba(0, 0, 0, 0.1);
|
162 |
+
overflow-y: scroll;
|
163 |
+
max-height: 75%;
|
164 |
+
scrollbar-width: none;
|
165 |
+
z-index: -100;
|
166 |
}
|
167 |
}
|
168 |
|
|
|
193 |
position: -webkit-sticky; /* For Safari */
|
194 |
position: sticky;
|
195 |
top: 10px; /* Adjust this value if needed */
|
196 |
+
overflow-y: scroll;
|
197 |
+
max-height: 75%;
|
198 |
+
scrollbar-width: none;
|
199 |
+
z-index: -100;
|
200 |
}
|
201 |
}
|
202 |
|