DeepSeek/DeepSeek-Coder
1
0
Fork 0
mirror of https://github.com/deepseek-ai/DeepSeek-Coder synced 2024-12-04 18:14:44 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update instruct model evaluation in README.md

Browse Source
This commit is contained in:
DejianYang 2023-11-10 17:47:42 +08:00 committed by GitHub
parent e957b8e651
commit cae2ab3765
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 11 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
Evaluation/HumanEval/README.md
View File

@ -27,7 +27,18 @@ LANGUAGE="python"
python -m accelerate.commands.launch --config_file test_config.yaml eval_pal.py --logdir ${MODEL_NAME_OR_PATH} --language ${LANGUAGE} --dataroot ${DATASET_ROOT}
```
To evaluate the instruction-based model, please follow the script below:
```bash
LANG="python"
OUPUT_DIR="output"
MODEL="deepseek-coder-33b-instruct"
CUDA_VISIBLE_DEVICES=0,1 python eval_instruct.py \
--model "deepseek-ai/$MODEL" \
--output_path "$OUPUT_DIR/${LANG}.$MODEL.jsonl" \
--language $LANG \
--temp_dir $OUPUT_DIR
```
## 4. Experimental Results