Update README.md
Browse files
README.md
CHANGED
|
@@ -87,9 +87,10 @@ Code Shell uses GPT-2 as its foundational architecture and incorporates technolo
|
|
| 87 |
We selected the two most popular code evaluation datasets currently available (HumanEval and MBPP) to assess the model. Compared to the two most advanced 7b LLM for code, CodeLllama and Starcoder, Codeshell achieved the best results. The specific evaluation results are as follows.
|
| 88 |
|
| 89 |
### Pass@1
|
| 90 |
-
| 任务 |
|
| 91 |
| ------- | --------- | --------- | --------- |
|
| 92 |
| humaneval | **34.32** | 29.44 | 27.80 |
|
|
|
|
| 93 |
| multiple-js | **33.17** | 31.30 | 27.02 |
|
| 94 |
| multiple-java | **30.43** | 29.24 | 24.30 |
|
| 95 |
| multiple-cpp | **28.21** | 27.33 | 23.04 |
|
|
|
|
| 87 |
We selected the two most popular code evaluation datasets currently available (HumanEval and MBPP) to assess the model. Compared to the two most advanced 7b LLM for code, CodeLllama and Starcoder, Codeshell achieved the best results. The specific evaluation results are as follows.
|
| 88 |
|
| 89 |
### Pass@1
|
| 90 |
+
| 任务 | iter_0000230 | codellama | starcoder7b |
|
| 91 |
| ------- | --------- | --------- | --------- |
|
| 92 |
| humaneval | **34.32** | 29.44 | 27.80 |
|
| 93 |
+
| mbpp | **38.65** | 37.60 | 34.16 |
|
| 94 |
| multiple-js | **33.17** | 31.30 | 27.02 |
|
| 95 |
| multiple-java | **30.43** | 29.24 | 24.30 |
|
| 96 |
| multiple-cpp | **28.21** | 27.33 | 23.04 |
|