diff --git a/README.md b/README.md index 3149b87..fc0bbb6 100644 --- a/README.md +++ b/README.md @@ -325,15 +325,38 @@ The reproducible code for the following evaluation results can be found in the [ #### 4) Program-Aid Math Reasoning Benchmark ![Math](pictures/Math.png) -### 7. Resources +### 7. Q&A + +#### Could You Provide the tokenizer.model File for GGUF Model Quantization? + +DeepSeek Coder utilizes the [HuggingFace Tokenizer](https://huggingface.co/docs/tokenizers/index) to implement the Bytelevel-BPE algorithm, with specially designed pre-tokenizers to ensure optimal performance. Currently, there is no direct way to convert the tokenizer into a SentencePiece tokenizer. We have submitted a [PR](https://github.com/ggerganov/llama.cpp/pull/4070) to the popular quantization repository [llama.cpp](https://github.com/ggerganov/llama.cpp) to fully support all HuggingFace pre-tokenizers, including ours. + +While waiting for the PR to be merged, you can generate your GGUF model using the following steps: + +```bash +git clone https://github.com/DOGEwbx/llama.cpp.git +cd llama.cpp +git checkout regex_gpt2_preprocess +# set up the environment according to README +make +python3 -m pip install -r requirements.txt +# generate GGUF model +python convert-hf-to-gguf.py --outfile --model-name deepseekcoder +# use q4_0 quantization as an example +./quantize q4_0 +./main -m -n 128 -p +``` + + +### 8. Resources [awesome-deepseek-coder](https://github.com/deepseek-ai/awesome-deepseek-coder) is a curated list of open-source projects related to DeepSeek Coder. -### 8. License +### 9. License This code repository is licensed under the MIT License. The use of DeepSeek Coder models is subject to the Model License. DeepSeek Coder supports commercial use. See the [LICENSE-CODE](LICENSE-CODE) and [LICENSE-MODEL](LICENSE-MODEL) for more details. -### 9. Citation +### 10. Citation ``` @misc{deepseek-coder, author = {DeepSeek AI}, @@ -345,6 +368,6 @@ See the [LICENSE-CODE](LICENSE-CODE) and [LICENSE-MODEL](LICENSE-MODEL) for more } ``` -### 10. Contact +### 11. Contact If you have any questions, please raise an issue or contact us at [agi_code@deepseek.com](mailto:agi_code@deepseek.com).