DeepSeek/DualPipe
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DualPipe synced 2025-04-07 22:14:13 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Fix bubble of 1F1B

Browse Source
This commit is contained in:
Weber Xie 2025-02-27 10:25:33 +08:00 committed by Jiashi Li
parent fbe0ac0d6e
commit dbb267dd85
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -15,7 +15,7 @@ have mutually overlapped computation and communication
| Method | Bubble | Parameter | Activation | | Method | Bubble | Parameter | Activation |
|-------------|---------------------------------|-----------|------------| |-------------|---------------------------------|-----------|------------|
| 1F1B | (PP-1)(𝐹+𝑊) | 1× | PP | | 1F1B | (PP-1)(𝐹+B) | 1× | PP |
| ZB1P | (PP-1)(𝐹+𝐵-2𝑊) | 1× | PP | | ZB1P | (PP-1)(𝐹+𝐵-2𝑊) | 1× | PP |
| DualPipe | (PP/2-1)(𝐹&𝐵+𝐵-3𝑊) | 2× | PP+1 | | DualPipe | (PP/2-1)(𝐹&𝐵+𝐵-3𝑊) | 2× | PP+1 |