DeepSeek/EPLB
1
0
Fork 0
mirror of https://github.com/deepseek-ai/EPLB synced 2025-06-26 18:15:49 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Fix grammatical error in strategy description.

Browse Source
This commit is contained in:
Shaoyuan CHEN
2025-02-27 17:38:04 +08:00
committed by GitHub
parent 9d303fafea 636fdc0a3e
commit 3079d71e1f
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@@ -2,7 +2,7 @@
When using expert parallelism (EP), different experts are assigned to different GPUs. Because the load of different When using expert parallelism (EP), different experts are assigned to different GPUs. Because the load of different
experts may vary depending on the current workload, it is important to keep the load of different GPUs balanced. experts may vary depending on the current workload, it is important to keep the load of different GPUs balanced.
As described in the DeepSeek-V3 paper, we adopt **redundant experts** strategy that duplicates heavy-loaded experts. As described in the DeepSeek-V3 paper, we adopt a **redundant experts** strategy that duplicates heavy-loaded experts.
Then, we heuristically pack the duplicated experts to GPUs to ensure load balancing across different GPUs. Moreover, Then, we heuristically pack the duplicated experts to GPUs to ensure load balancing across different GPUs. Moreover,
thanks to the **group-limited expert routing** used in DeepSeek-V3, we also attempt to place the experts of the same thanks to the **group-limited expert routing** used in DeepSeek-V3, we also attempt to place the experts of the same
group to the same node to reduce inter-node data traffic, whenever possible. group to the same node to reduce inter-node data traffic, whenever possible.