From b945d8135e85b73286b1735c9b935d2a46fc825d Mon Sep 17 00:00:00 2001 From: Fuli Luo Date: Fri, 21 Jun 2024 14:17:27 +0800 Subject: [PATCH] Update README.md --- README.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/README.md b/README.md index a28af9c..a2bdf62 100644 --- a/README.md +++ b/README.md @@ -268,6 +268,13 @@ Assistant: {assistant_message_1}<|end▁of▁sentence|>User: {user_message_2 Assistant: ``` +In the last round of dialogue, note that "Assistant:" has no space after the colon. Adding a space might cause the following issues on the 16B-Lite model: +- English questions receiving Chinese responses. +- Responses containing garbled text. +- Responses repeating excessively. + +Older versions of Ollama had this bug (see https://github.com/deepseek-ai/DeepSeek-Coder-V2/issues/12), but it has been fixed in the latest version. + ### Inference with vLLM (recommended) To utilize [vLLM](https://github.com/vllm-project/vllm) for model inference, please merge this Pull Request into your vLLM codebase: https://github.com/vllm-project/vllm/pull/4650.