DeepSeek/DeepSeek-R1
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-R1 synced 2025-01-22 09:45:30 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

docs: serving DeepSeek-R1-Distill models with SGLang

Browse Source
This commit is contained in:
Yineng Zhang 2025-01-20 20:30:16 +08:00 committed by GitHub
parent 23807ced51
commit f6948c24a8
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 6 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@ -191,6 +191,12 @@ For instance, you can easily start a service using [vLLM](https://github.com/vll
vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B --tensor-parallel-size 2 --max-model-len 32768 --enforce-eager vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B --tensor-parallel-size 2 --max-model-len 32768 --enforce-eager
``` ```
You can also easily start a service using [SGLang](https://github.com/sgl-project/sglang)
```bash
python3 -m sglang.launch_server --model deepseek-ai/DeepSeek-R1-Distill-Qwen-32B --trust-remote-code --tp 2
```
**NOTE: We recommend setting an appropriate temperature (between 0.5 and 0.7) when running these models, otherwise you may encounter issues with endless repetition or incoherent output.** **NOTE: We recommend setting an appropriate temperature (between 0.5 and 0.7) when running these models, otherwise you may encounter issues with endless repetition or incoherent output.**
## 7. License ## 7. License