Hugging Face's logo

FallenMerick
/
Smart-Lemon-Cookie-7B

Text Generation
Transformers
Safetensors
English
mistral
mergekit
Merge
roleplay
conversational
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Community
1
FallenMerick leaderboard-pr-bot commited on
Commit
69999bf
•
1 Parent(s): 367981b

Adding Evaluation Results (#1)

Browse files

- Adding Evaluation Results (34a133f9efea05f4d4edd40b6d28aa8ea87b6bf4)


Co-authored-by: Open LLM Leaderboard PR Bot <[email protected]>

Files changed (1) hide show
  1. README.md +123 -7
README.md CHANGED
@@ -1,12 +1,7 @@
1
  ---
2
- license: cc-by-4.0
3
  language:
4
  - en
5
- base_model:
6
- - SanjiWatsuki/Silicon-Maid-7B
7
- - MTSAIR/multi_verse_model
8
- - SanjiWatsuki/Kunoichi-7B
9
- - KatyTheCutie/LemonadeRP-4.5.3
10
  library_name: transformers
11
  tags:
12
  - mergekit
@@ -14,7 +9,114 @@ tags:
14
  - mistral
15
  - text-generation
16
  - roleplay
17
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  ---
19
 
20
  ![cute](https://huggingface.co/FallenMerick/Smart-Lemon-Cookie-7B/resolve/main/smart-lemon-cookie-cute.png)
@@ -66,3 +168,17 @@ parameters:
66
  dtype: float16
67
 
68
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
2
  language:
3
  - en
4
+ license: cc-by-4.0
 
 
 
 
5
  library_name: transformers
6
  tags:
7
  - mergekit
 
9
  - mistral
10
  - text-generation
11
  - roleplay
12
+ base_model:
13
+ - SanjiWatsuki/Silicon-Maid-7B
14
+ - MTSAIR/multi_verse_model
15
+ - SanjiWatsuki/Kunoichi-7B
16
+ - KatyTheCutie/LemonadeRP-4.5.3
17
+ model-index:
18
+ - name: Smart-Lemon-Cookie-7B
19
+ results:
20
+ - task:
21
+ type: text-generation
22
+ name: Text Generation
23
+ dataset:
24
+ name: AI2 Reasoning Challenge (25-Shot)
25
+ type: ai2_arc
26
+ config: ARC-Challenge
27
+ split: test
28
+ args:
29
+ num_few_shot: 25
30
+ metrics:
31
+ - type: acc_norm
32
+ value: 66.3
33
+ name: normalized accuracy
34
+ source:
35
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=FallenMerick/Smart-Lemon-Cookie-7B
36
+ name: Open LLM Leaderboard
37
+ - task:
38
+ type: text-generation
39
+ name: Text Generation
40
+ dataset:
41
+ name: HellaSwag (10-Shot)
42
+ type: hellaswag
43
+ split: validation
44
+ args:
45
+ num_few_shot: 10
46
+ metrics:
47
+ - type: acc_norm
48
+ value: 85.53
49
+ name: normalized accuracy
50
+ source:
51
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=FallenMerick/Smart-Lemon-Cookie-7B
52
+ name: Open LLM Leaderboard
53
+ - task:
54
+ type: text-generation
55
+ name: Text Generation
56
+ dataset:
57
+ name: MMLU (5-Shot)
58
+ type: cais/mmlu
59
+ config: all
60
+ split: test
61
+ args:
62
+ num_few_shot: 5
63
+ metrics:
64
+ - type: acc
65
+ value: 64.69
66
+ name: accuracy
67
+ source:
68
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=FallenMerick/Smart-Lemon-Cookie-7B
69
+ name: Open LLM Leaderboard
70
+ - task:
71
+ type: text-generation
72
+ name: Text Generation
73
+ dataset:
74
+ name: TruthfulQA (0-shot)
75
+ type: truthful_qa
76
+ config: multiple_choice
77
+ split: validation
78
+ args:
79
+ num_few_shot: 0
80
+ metrics:
81
+ - type: mc2
82
+ value: 60.66
83
+ source:
84
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=FallenMerick/Smart-Lemon-Cookie-7B
85
+ name: Open LLM Leaderboard
86
+ - task:
87
+ type: text-generation
88
+ name: Text Generation
89
+ dataset:
90
+ name: Winogrande (5-shot)
91
+ type: winogrande
92
+ config: winogrande_xl
93
+ split: validation
94
+ args:
95
+ num_few_shot: 5
96
+ metrics:
97
+ - type: acc
98
+ value: 77.74
99
+ name: accuracy
100
+ source:
101
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=FallenMerick/Smart-Lemon-Cookie-7B
102
+ name: Open LLM Leaderboard
103
+ - task:
104
+ type: text-generation
105
+ name: Text Generation
106
+ dataset:
107
+ name: GSM8k (5-shot)
108
+ type: gsm8k
109
+ config: main
110
+ split: test
111
+ args:
112
+ num_few_shot: 5
113
+ metrics:
114
+ - type: acc
115
+ value: 54.06
116
+ name: accuracy
117
+ source:
118
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=FallenMerick/Smart-Lemon-Cookie-7B
119
+ name: Open LLM Leaderboard
120
  ---
121
 
122
  ![cute](https://huggingface.co/FallenMerick/Smart-Lemon-Cookie-7B/resolve/main/smart-lemon-cookie-cute.png)
 
168
  dtype: float16
169
 
170
  ```
171
+
172
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
173
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_FallenMerick__Smart-Lemon-Cookie-7B)
174
+
175
+ | Metric |Value|
176
+ |---------------------------------|----:|
177
+ |Avg. |68.16|
178
+ |AI2 Reasoning Challenge (25-Shot)|66.30|
179
+ |HellaSwag (10-Shot) |85.53|
180
+ |MMLU (5-Shot) |64.69|
181
+ |TruthfulQA (0-shot) |60.66|
182
+ |Winogrande (5-shot) |77.74|
183
+ |GSM8k (5-shot) |54.06|
184
+