Spaces:
Running
Running
Description is updates. Models are sorted by default based on accuracy/average performance.
Browse files
app.py
CHANGED
@@ -9,17 +9,18 @@ TITLE = '''<h1>
|
|
9 |
</h1>'''
|
10 |
INTRODUCTION_TEXT = '''
|
11 |
Evaluating the chat, safety, reasoning, and translation capabilities of Reward Models in Turkish. This space is a more detailed version of [M-RewardBench](https://huggingface.co/spaces/C4AI-Community/m-rewardbench) for Turkish
|
12 |
-
You can find more details (paper,code,etc.) on their space. This space uses the Turkish subset of [C4AI-Community/multilingual-reward-bench](https://hf.co/datasets/C4AI-Community/multilingual-reward-bench)
|
13 |
-
I want to thank them for relasing this dataset 🤗.
|
14 |
|
15 |
Most of the current models were evaluated with max token lenght of 2048. This effects the performance since it can cut some of the text. So if you try replicating the results with higher token size
|
16 |
you may get slightly better results (which also depends on the model).
|
17 |
|
|
|
|
|
18 |
For the description of subsets you can check out the about section of the original [space](https://huggingface.co/spaces/allenai/reward-bench).
|
19 |
|
20 |
-
### Important warning
|
21 |
In the original [English version of the dataset](https://huggingface.co/spaces/allenai/reward-bench) it is noted that some of the models are
|
22 |
-
unintentionally contaminated. You can find more on [here](https://gist.github.com/natolambert/1aed306000c13e0e8c5bc17c1a5dd300). I
|
23 |
warn anyways.
|
24 |
|
25 |
'''
|
@@ -144,7 +145,7 @@ with demo:
|
|
144 |
with gr.Tabs() as tabs:
|
145 |
with gr.TabItem("🏅 Subset performance"):
|
146 |
df = get_result_data()
|
147 |
-
|
148 |
|
149 |
numeric_cols = df.select_dtypes(include=[np.number]).columns
|
150 |
global_min = df.select_dtypes(include='number').min().min()#.astype(float)
|
@@ -165,6 +166,7 @@ with demo:
|
|
165 |
|
166 |
with gr.TabItem("🏅 Categorical"):
|
167 |
df = get_categorical_data()
|
|
|
168 |
|
169 |
numeric_cols = df.select_dtypes(include=[np.number]).columns
|
170 |
|
|
|
9 |
</h1>'''
|
10 |
INTRODUCTION_TEXT = '''
|
11 |
Evaluating the chat, safety, reasoning, and translation capabilities of Reward Models in Turkish. This space is a more detailed version of [M-RewardBench](https://huggingface.co/spaces/C4AI-Community/m-rewardbench) for Turkish
|
12 |
+
You can find more details (paper,code,etc.) on their space. This space uses the Turkish subset of [C4AI-Community/multilingual-reward-bench](https://hf.co/datasets/C4AI-Community/multilingual-reward-bench), I want to thank them for relasing this dataset 🤗.
|
|
|
13 |
|
14 |
Most of the current models were evaluated with max token lenght of 2048. This effects the performance since it can cut some of the text. So if you try replicating the results with higher token size
|
15 |
you may get slightly better results (which also depends on the model).
|
16 |
|
17 |
+
Due to resource limit these results are just for a single run of each model. Running each model multiple times and taking the mean would give better representation of the actual performance.
|
18 |
+
|
19 |
For the description of subsets you can check out the about section of the original [space](https://huggingface.co/spaces/allenai/reward-bench).
|
20 |
|
21 |
+
### Important warning ⚠️:
|
22 |
In the original [English version of the dataset](https://huggingface.co/spaces/allenai/reward-bench) it is noted that some of the models are
|
23 |
+
unintentionally contaminated. You can find more on [here](https://gist.github.com/natolambert/1aed306000c13e0e8c5bc17c1a5dd300). I doubt that models can generalize enough to have a performance boost even if they are trained with the English translation of a dataset but I just wanted to
|
24 |
warn anyways.
|
25 |
|
26 |
'''
|
|
|
145 |
with gr.Tabs() as tabs:
|
146 |
with gr.TabItem("🏅 Subset performance"):
|
147 |
df = get_result_data()
|
148 |
+
df = df.sort_values(by="accuracy", ascending=False)
|
149 |
|
150 |
numeric_cols = df.select_dtypes(include=[np.number]).columns
|
151 |
global_min = df.select_dtypes(include='number').min().min()#.astype(float)
|
|
|
166 |
|
167 |
with gr.TabItem("🏅 Categorical"):
|
168 |
df = get_categorical_data()
|
169 |
+
df = df.sort_values(by="Average", ascending=False)
|
170 |
|
171 |
numeric_cols = df.select_dtypes(include=[np.number]).columns
|
172 |
|