Spaces:

inflaton-ai
/

logical-reasoning

Build error

inflaton commited on Sep 20, 2024

Commit

5158692

1 Parent(s): a8683cf

mistral 5-shot

Files changed (3) hide show

data/Mistral-7B-v0.3-Chinese-Chat_results.csv CHANGED Viewed

The diff for this file is too large to render. See raw diff

llm_toolkit/eval_shots.py CHANGED Viewed

@@ -99,7 +99,7 @@ def evaluate_model_with_num_shots(
     model_name,
     data_path,
     start_num_shots=0,
-    range_num_shots=[0, 5],
     batch_size=1,
     max_new_tokens=2048,
     device="cuda",

     model_name,
     data_path,
     start_num_shots=0,
+    range_num_shots=[0, 5, 10, 20, 30, 40, 50],
     batch_size=1,
     max_new_tokens=2048,
     device="cuda",

scripts/eval-mgtv-shots.sh CHANGED Viewed

@@ -13,7 +13,7 @@ cat /etc/os-release
 lscpu
 grep MemTotal /proc/meminfo
-export START_NUM_SHOTS=10
 $BASEDIR/scripts/eval-shots.sh shenzhi-wang Mistral-7B-v0.3-Chinese-Chat

 lscpu
 grep MemTotal /proc/meminfo
+export START_NUM_SHOTS=5
 $BASEDIR/scripts/eval-shots.sh shenzhi-wang Mistral-7B-v0.3-Chinese-Chat