diff --git a/README.md b/README.md index 7beb8de..f708570 100644 --- a/README.md +++ b/README.md @@ -3,44 +3,35 @@
- DeepSeek LLM + DeepSeek LLM
- Homepage + Homepage - Chat + Chat - Hugging Face + Hugging Face -
- -
- - Discord + Discord - - Wechat + + Wechat - Twitter Follow + Twitter Follow - -
- -
- - Code License + Code License - Model License + Model License
@@ -64,30 +55,37 @@ Today, we’re introducing DeepSeek-V2, a strong Mixture-of-Experts (MoE) language model characterized by economical training and efficient inference. It comprises 236B total parameters, of which 21B are activated for each token. Compared with DeepSeek 67B, DeepSeek-V2 achieves stronger performance, and meanwhile saves 42.5% of training costs, reduces the KV cache by 93.3%, and boosts the maximum generation throughput to 5.76 times.

-

- - + +

+ We pretrained DeepSeek-V2 on a diverse and high-quality corpus comprising 8.1 trillion tokens. This comprehensive pretraining was followed by a process of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) to fully unleash the model's capabilities. The evaluation results validate the effectiveness of our approach as DeepSeek-V2 achieves remarkable performance on both standard benchmarks and open-ended generation evaluation. -## 2. Model Downloads +## 2. News + +- 2024.05.16: We released the DeepSeek-V2-Lite. +- 2024.05.06: We released the DeepSeek-V2. + +## 3. Model Downloads
-| **Model** | **Context Length** | **Download** | -| :------------: | :------------: | :------------: | -| DeepSeek-V2 | 128k | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2) | -| DeepSeek-V2-Chat (RL) | 128k | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2-Chat) | +| **Model** | **#Total Params** | **#Activated Params** | **Context Length** | **Download** | +| :------------: | :------------: | :------------: | :------------: | :------------: | +| DeepSeek-V2-Lite | 16B | 2.4B | 32k | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite) | +| DeepSeek-V2-Lite-Chat (SFT) | 16B | 2.4B | 32k | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat) | +| DeepSeek-V2 | 236B | 21B | 128k | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2) | +| DeepSeek-V2-Chat (RL) | 236B | 21B | 128k | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2-Chat) |
Due to the constraints of HuggingFace, the open-source code currently experiences slower performance than our internal codebase when running on GPUs with Huggingface. To facilitate the efficient execution of our model, we offer a dedicated vllm solution that optimizes performance for running our model effectively. -## 3. Evaluation Results +## 4. Evaluation Results ### Base Model -#### Standard Benchmark +#### Standard Benchmark (Models larger than 67B)
@@ -102,18 +100,35 @@ Due to the constraints of HuggingFace, the open-source code currently experience | **GSM8K** | Math | 83.0 | 80.3 | 63.4 | 79.2 | | **Math** | Math | 42.2 | 42.5 | 18.7 | 43.6 | +
+ +#### Standard Benchmark (Models smaller than 16B) +
+ +| **Benchmark** | **Domain** | **DeepSeek 7B (Dense)** | **DeepSeekMoE 16B** | **DeepSeek-V2-Lite (MoE-16B)** | +|:-------------:|:----------:|:--------------:|:-----------------:|:--------------------------:| +| **Architecture** | - | MHA+Dense | MHA+MoE | MLA+MoE | +| **MMLU** | English | 48.2 | 45.0 | 58.3 | +| **BBH** | English | xxxx | xxxx | 44.1 | +| **C-Eval** | Chinese | 45.0 | 40.6 | 60.3 | +| **CMMLU** | Chinese | 47.2 | 42.5 | 64.3 | +| **HumanEval** | Code | 26.2 | 26.8 | 29.9 | +| **MBPP** | Code | 39.0 | 39.2 | 43.2 | +| **GSM8K** | Math | 17.4 | 18.8 | 41.1 | +| **Math** | Math | 3.3 | 4.3 | 17.1 | +
For more evaluation details, such as few-shot settings and prompts, please check our paper. #### Context Window

- +

Evaluation results on the ``Needle In A Haystack`` (NIAH) tests. DeepSeek-V2 performs well across all context window lengths up to **128K**. ### Chat Model -#### Standard Benchmark +#### Standard Benchmark (Models larger than 67B)
| Benchmark | Domain | QWen1.5 72B Chat | Mixtral 8x22B | LLaMA3 70B Instruct | DeepSeek-V1 Chat (SFT) | DeepSeek-V2 Chat (SFT) | DeepSeek-V2 Chat (RL) | @@ -130,10 +145,27 @@ Evaluation results on the ``Needle In A Haystack`` (NIAH) tests. DeepSeek-V2 pe
+#### Standard Benchmark (Models smaller than 16B) + +
+ +| Benchmark | Domain | DeepSeek 7B Chat (SFT) | DeepSeekMoE 16B Chat (SFT) | DeepSeek-V2-Lite 16B Chat (SFT) | +|:-----------:|:----------------:|:------------------:|:---------------:|:---------------------:| +| **MMLU** | English | 49.7 | 47.2 | 55.7 | +| **BBH** | English | 43.1 | 42.2 | 48.1 | +| **C-Eval** | Chinese | 44.7 | 40.0 | 60.1 | +| **CMMLU** | Chinese | 51.2 | 49.3 | 62.5 | +| **HumanEval** | Code | 45.1 | 45.7 | 57.3 | +| **MBPP** | Code | 39.0 | 46.2 | 45.8 | +| **GSM8K** | Math | 62.6 | 62.2 | 72.0 | +| **Math** | Math | 14.7 | 15.2 | 27.9 | + +
+ #### English Open Ended Generation Evaluation We evaluate our model on AlpacaEval 2.0 and MTBench, showing the competitive performance of DeepSeek-V2-Chat-RL on English conversation generation.

- +

#### Chinese Open Ended Generation Evaluation @@ -153,6 +185,7 @@ We evaluate our model on AlpacaEval 2.0 and MTBench, showing the competitive per | DeepSeek-67B-Chat | 开源 | 6.43 | 5.75 | 7.11 | | Yi-34B-Chat (零一万物) | 开源 | 6.12 | 4.86 | 7.38 | | gpt-3.5-turbo-0613 | 闭源 | 6.08 | 5.35 | 6.71 | +| DeepSeek-V2-Lite 16B Chat | 开源 | 6.01 | 4.71 | 7.32 | @@ -160,31 +193,29 @@ We evaluate our model on AlpacaEval 2.0 and MTBench, showing the competitive per We evaluate our model on LiveCodeBench (0901-0401), a benchmark designed for live coding challenges. As illustrated, DeepSeek-V2 demonstrates considerable proficiency in LiveCodeBench, achieving a Pass@1 score that surpasses several other sophisticated models. This performance highlights the model's effectiveness in tackling live coding tasks.

- +

-## 4. Model Architecture +## 5. Model Architecture DeepSeek-V2 adopts innovative architectures to guarantee economical training and efficient inference： - For attention, we design MLA (Multi-head Latent Attention), which utilizes low-rank key-value union compression to eliminate the bottleneck of inference-time key-value cache, thus supporting efficient inference. - For Feed-Forward Networks (FFNs), we adopt DeepSeekMoE architecture, a high-performance MoE architecture that enables training stronger models at lower costs.

- +

- -## 5. Chat Website +## 6. Chat Website You can chat with the DeepSeek-V2 on DeepSeek's official website: [chat.deepseek.com](https://chat.deepseek.com/sign_in) -## 6. API Platform +## 7. API Platform We also provide OpenAI-Compatible API at DeepSeek Platform: [platform.deepseek.com](https://platform.deepseek.com/). Sign up for over millions of free tokens. And you can also pay-as-you-go at an unbeatable price.

- +

- -## 7. How to run locally +## 8. How to run locally **To utilize DeepSeek-V2 in BF16 format for inference, 80GB*8 GPUs are required.** ### Inference with Huggingface's Transformers You can directly employ [Huggingface's Transformers](https://github.com/huggingface/transformers) for model inference. @@ -299,10 +330,10 @@ llm = ChatOpenAI( temperature=0.85, max_tokens=8000) ``` -## 8. License +## 9. License This code repository is licensed under [the MIT License](LICENSE-CODE). The use of DeepSeek-V2 Base/Chat models is subject to [the Model License](LICENSE-MODEL). DeepSeek-V2 series (including Base and Chat) supports commercial use. -## 9. Citation +## 10. Citation ``` @misc{deepseekv2, title={DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model}, @@ -314,5 +345,5 @@ This code repository is licensed under [the MIT License](LICENSE-CODE). The use } ``` -## 10. Contact +## 11. Contact If you have any questions, please raise an issue or contact us at [service@deepseek.com](service@deepseek.com).