DeepSeek/DeepSeek-Coder
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-Coder synced 2024-12-05 02:24:46 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update README.md

Browse Source 
add notes on ChatLLM.cpp
This commit is contained in:
Judd 2023-12-09 18:05:57 +08:00 committed by GitHub
parent 24add12231
commit 96a277d5c1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
README.md
View File

@ -350,6 +350,9 @@ python convert-hf-to-gguf.py <MODEL_PATH> --outfile <GGUF_PATH> --model-name dee
./quantize <GGUF_PATH> <OUTPUT_PATH> q4_0
./main -m <OUTPUT_PATH> -n 128 -p <PROMPT>
```
You can also try out other [ggml](https://github.com/ggerganov/ggml)-based inferencers, such as [ChatLLM.cpp](https://github.com/foldl/chatllm.cpp), as well.
##### GPTQ(exllamav2)
`UPDATE:`[exllamav2](https://github.com/turboderp/exllamav2) has been able to support Huggingface Tokenizer. Please pull the latest version and try out.