From c14b7f723bd32c4db5492cbb2731589ec884b710 Mon Sep 17 00:00:00 2001
From: BingxuanWang <151806429+BingxuanWang@users.noreply.github.com>
Date: Thu, 23 Nov 2023 15:05:18 +0800
Subject: [PATCH] Update Q&A in README.md

---
 README.md | 31 +++++++++++++++++++++++++++----
 1 file changed, 27 insertions(+), 4 deletions(-)

diff --git a/README.md b/README.md
index 3149b87..fc0bbb6 100644
--- a/README.md
+++ b/README.md
@@ -325,15 +325,38 @@ The reproducible code for the following evaluation results can be found in the [
 #### 4) Program-Aid Math Reasoning Benchmark
 ![Math](pictures/Math.png)
 
-### 7. Resources
+### 7. Q&A
+
+#### Could You Provide the tokenizer.model File for GGUF Model Quantization?
+
+DeepSeek Coder utilizes the [HuggingFace Tokenizer](https://huggingface.co/docs/tokenizers/index) to implement the Bytelevel-BPE algorithm, with specially designed pre-tokenizers to ensure optimal performance. Currently, there is no direct way to convert the tokenizer into a SentencePiece tokenizer. We have submitted a [PR](https://github.com/ggerganov/llama.cpp/pull/4070) to the popular quantization repository [llama.cpp](https://github.com/ggerganov/llama.cpp) to fully support all HuggingFace pre-tokenizers, including ours.
+
+While waiting for the PR to be merged, you can generate your GGUF model using the following steps:
+
+```bash
+git clone https://github.com/DOGEwbx/llama.cpp.git
+cd llama.cpp
+git checkout regex_gpt2_preprocess
+# set up the environment according to README
+make
+python3 -m pip install -r requirements.txt
+# generate GGUF model
+python convert-hf-to-gguf.py <MODEL_PATH> --outfile <GGUF_PATH> --model-name deepseekcoder
+# use q4_0 quantization as an example
+./quantize <GGUF_PATH> <OUTPUT_PATH> q4_0
+./main -m <OUTPUT_PATH> -n 128 -p <PROMPT>
+```
+
+
+### 8. Resources
 [awesome-deepseek-coder](https://github.com/deepseek-ai/awesome-deepseek-coder) is a curated list of open-source projects related to DeepSeek Coder.
 
-### 8. License
+### 9. License
 This code repository is licensed under the MIT License. The use of DeepSeek Coder models is subject to the Model License. DeepSeek Coder supports commercial use.
 
 See the [LICENSE-CODE](LICENSE-CODE) and [LICENSE-MODEL](LICENSE-MODEL) for more details.
 
-### 9. Citation
+### 10. Citation
 ```
 @misc{deepseek-coder,
   author = {DeepSeek AI},
@@ -345,6 +368,6 @@ See the [LICENSE-CODE](LICENSE-CODE) and [LICENSE-MODEL](LICENSE-MODEL) for more
 }
 ```
 
-### 10. Contact
+### 11. Contact
 
 If you have any questions, please raise an issue or contact us at [agi_code@deepseek.com](mailto:agi_code@deepseek.com).