Spaces:
Running
Running
comms-figures (#52)
Browse files- update figures (a30d19d32a71110d5bf3068f4aff054df40adece)
assets/images/5D_nutshell_tp_sp.svg
CHANGED
|
|
assets/images/5d_nutshell_cp.svg
CHANGED
|
|
assets/images/5d_nutshell_ep.svg
CHANGED
|
|
dist/assets/images/5D_nutshell_tp_sp.svg
CHANGED
|
|
dist/assets/images/5d_nutshell_cp.svg
CHANGED
|
|
dist/assets/images/5d_nutshell_ep.svg
CHANGED
|
|
dist/index.html
CHANGED
@@ -1648,7 +1648,7 @@
|
|
1648 |
|
1649 |
<p><strong>Tensor Parallelism</strong> (with Sequence Parallelism) is naturally complementary and can be combined with both Pipeline Parallelism and ZeRO-3 as it relies on the distributive property of matrix multiplications which allows weights and activations to be sharded and computed independently before being combined.</p>
|
1650 |
|
1651 |
-
<img alt="TP & SP diagram" src="/assets/images/
|
1652 |
<!-- <p><img alt="image.png" src="/assets/images/placeholder.png" /></p> -->
|
1653 |
|
1654 |
|
|
|
1648 |
|
1649 |
<p><strong>Tensor Parallelism</strong> (with Sequence Parallelism) is naturally complementary and can be combined with both Pipeline Parallelism and ZeRO-3 as it relies on the distributive property of matrix multiplications which allows weights and activations to be sharded and computed independently before being combined.</p>
|
1650 |
|
1651 |
+
<img alt="TP & SP diagram" src="/assets/images/5d_nutshell_tp_sp.svg" style="width: 1000px; max-width: none;" />
|
1652 |
<!-- <p><img alt="image.png" src="/assets/images/placeholder.png" /></p> -->
|
1653 |
|
1654 |
|
src/index.html
CHANGED
@@ -1648,7 +1648,7 @@
|
|
1648 |
|
1649 |
<p><strong>Tensor Parallelism</strong> (with Sequence Parallelism) is naturally complementary and can be combined with both Pipeline Parallelism and ZeRO-3 as it relies on the distributive property of matrix multiplications which allows weights and activations to be sharded and computed independently before being combined.</p>
|
1650 |
|
1651 |
-
<img alt="TP & SP diagram" src="/assets/images/
|
1652 |
<!-- <p><img alt="image.png" src="/assets/images/placeholder.png" /></p> -->
|
1653 |
|
1654 |
|
|
|
1648 |
|
1649 |
<p><strong>Tensor Parallelism</strong> (with Sequence Parallelism) is naturally complementary and can be combined with both Pipeline Parallelism and ZeRO-3 as it relies on the distributive property of matrix multiplications which allows weights and activations to be sharded and computed independently before being combined.</p>
|
1650 |
|
1651 |
+
<img alt="TP & SP diagram" src="/assets/images/5d_nutshell_tp_sp.svg" style="width: 1000px; max-width: none;" />
|
1652 |
<!-- <p><img alt="image.png" src="/assets/images/placeholder.png" /></p> -->
|
1653 |
|
1654 |
|