Nondzu commited on
Commit
1339c50
1 Parent(s): 0ea53f0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -0
README.md CHANGED
@@ -32,6 +32,25 @@ Below is an instruction that describes a task. Write a response that appropriate
32
  ```
33
 
34
  [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
35
  ## Settings:
36
  ```
37
  base_model: mistralai/Mistral-7B-Instruct-v0.1
 
32
  ```
33
 
34
  [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
35
+ ## eval plus
36
+ Human eval plus: https://github.com/evalplus/evalplus
37
+
38
+ ```
39
+ Nondzu mistral-7b-code
40
+ Base
41
+ {'pass@1': 0.3353658536585366}
42
+ Base + Extra
43
+ {'pass@1': 0.2804878048780488}
44
+ ```
45
+ to compare here is original Mistral model tested on the same machine
46
+ ```
47
+ Mistral 7b
48
+ Base
49
+ {'pass@1': 0.2926829268292683}
50
+ Base + Extra
51
+ {'pass@1': 0.24390243902439024}
52
+ ```
53
+
54
  ## Settings:
55
  ```
56
  base_model: mistralai/Mistral-7B-Instruct-v0.1