DeepSeek/DualPipe
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DualPipe synced 2025-04-30 02:20:13 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

fix the front of readme

Browse Source
This commit is contained in:
RK 2025-02-27 10:56:13 +08:00
parent 75531bb05e
commit 89f5de17a3
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -2,7 +2,7 @@
DualPipe is an innovative bidirectional pipeline parallism algorithm introduced in the [DeepSeek-V3 Technical Report](https://arxiv.org/pdf/2412.19437). It achieves full overlap of forward and backward computation-communication phases, also reducing pipeline bubbles. For detailed information on computation-communication overlap, please refer to the [profile data](https://github.com/deepseek-ai/profile-data). DualPipe is an innovative bidirectional pipeline parallism algorithm introduced in the [DeepSeek-V3 Technical Report](https://arxiv.org/pdf/2412.19437). It achieves full overlap of forward and backward computation-communication phases, also reducing pipeline bubbles. For detailed information on computation-communication overlap, please refer to the [profile data](https://github.com/deepseek-ai/profile-data).
### Schedules ## Schedules
![schedules](images/schedules.png) ![schedules](images/schedules.png)
@ -19,7 +19,7 @@ have mutually overlapped computation and communication
| ZB1P | (PP-1)(𝐹+𝐵-2𝑊) | 1× | PP | | ZB1P | (PP-1)(𝐹+𝐵-2𝑊) | 1× | PP |
| DualPipe | (PP/2-1)(𝐹&𝐵+𝐵-3𝑊) | 2× | PP+1 | | DualPipe | (PP/2-1)(𝐹&𝐵+𝐵-3𝑊) | 2× | PP+1 |
𝐹 denotes the execution time of a forward chunk, 𝐵 denotes the execution time of a **𝐹** denotes the execution time of a forward chunk, 𝐵 denotes the execution time of a
full backward chunk, 𝑊 denotes the execution time of a "backward for weights" chunk, and 𝐹&𝐵 full backward chunk, 𝑊 denotes the execution time of a "backward for weights" chunk, and 𝐹&𝐵
denotes the execution time of two mutually overlapped forward and backward chunks. denotes the execution time of two mutually overlapped forward and backward chunks.