kesimeg commited on
Commit
49901fd
·
verified ·
1 Parent(s): d8710b1

Description is updates. Models are sorted by default based on accuracy/average performance.

Browse files
Files changed (1) hide show
  1. app.py +7 -5
app.py CHANGED
@@ -9,17 +9,18 @@ TITLE = '''<h1>
9
  </h1>'''
10
  INTRODUCTION_TEXT = '''
11
  Evaluating the chat, safety, reasoning, and translation capabilities of Reward Models in Turkish. This space is a more detailed version of [M-RewardBench](https://huggingface.co/spaces/C4AI-Community/m-rewardbench) for Turkish
12
- You can find more details (paper,code,etc.) on their space. This space uses the Turkish subset of [C4AI-Community/multilingual-reward-bench](https://hf.co/datasets/C4AI-Community/multilingual-reward-bench)
13
- I want to thank them for relasing this dataset 🤗.
14
 
15
  Most of the current models were evaluated with max token lenght of 2048. This effects the performance since it can cut some of the text. So if you try replicating the results with higher token size
16
  you may get slightly better results (which also depends on the model).
17
 
 
 
18
  For the description of subsets you can check out the about section of the original [space](https://huggingface.co/spaces/allenai/reward-bench).
19
 
20
- ### Important warning: ⚠️,
21
  In the original [English version of the dataset](https://huggingface.co/spaces/allenai/reward-bench) it is noted that some of the models are
22
- unintentionally contaminated. You can find more on [here](https://gist.github.com/natolambert/1aed306000c13e0e8c5bc17c1a5dd300). I am doubt that models can generalize enough to have a performance boost even if they are trained with the English translation of a dataset but I just wanted to
23
  warn anyways.
24
 
25
  '''
@@ -144,7 +145,7 @@ with demo:
144
  with gr.Tabs() as tabs:
145
  with gr.TabItem("🏅 Subset performance"):
146
  df = get_result_data()
147
-
148
 
149
  numeric_cols = df.select_dtypes(include=[np.number]).columns
150
  global_min = df.select_dtypes(include='number').min().min()#.astype(float)
@@ -165,6 +166,7 @@ with demo:
165
 
166
  with gr.TabItem("🏅 Categorical"):
167
  df = get_categorical_data()
 
168
 
169
  numeric_cols = df.select_dtypes(include=[np.number]).columns
170
 
 
9
  </h1>'''
10
  INTRODUCTION_TEXT = '''
11
  Evaluating the chat, safety, reasoning, and translation capabilities of Reward Models in Turkish. This space is a more detailed version of [M-RewardBench](https://huggingface.co/spaces/C4AI-Community/m-rewardbench) for Turkish
12
+ You can find more details (paper,code,etc.) on their space. This space uses the Turkish subset of [C4AI-Community/multilingual-reward-bench](https://hf.co/datasets/C4AI-Community/multilingual-reward-bench), I want to thank them for relasing this dataset 🤗.
 
13
 
14
  Most of the current models were evaluated with max token lenght of 2048. This effects the performance since it can cut some of the text. So if you try replicating the results with higher token size
15
  you may get slightly better results (which also depends on the model).
16
 
17
+ Due to resource limit these results are just for a single run of each model. Running each model multiple times and taking the mean would give better representation of the actual performance.
18
+
19
  For the description of subsets you can check out the about section of the original [space](https://huggingface.co/spaces/allenai/reward-bench).
20
 
21
+ ### Important warning ⚠️:
22
  In the original [English version of the dataset](https://huggingface.co/spaces/allenai/reward-bench) it is noted that some of the models are
23
+ unintentionally contaminated. You can find more on [here](https://gist.github.com/natolambert/1aed306000c13e0e8c5bc17c1a5dd300). I doubt that models can generalize enough to have a performance boost even if they are trained with the English translation of a dataset but I just wanted to
24
  warn anyways.
25
 
26
  '''
 
145
  with gr.Tabs() as tabs:
146
  with gr.TabItem("🏅 Subset performance"):
147
  df = get_result_data()
148
+ df = df.sort_values(by="accuracy", ascending=False)
149
 
150
  numeric_cols = df.select_dtypes(include=[np.number]).columns
151
  global_min = df.select_dtypes(include='number').min().min()#.astype(float)
 
166
 
167
  with gr.TabItem("🏅 Categorical"):
168
  df = get_categorical_data()
169
+ df = df.sort_values(by="Average", ascending=False)
170
 
171
  numeric_cols = df.select_dtypes(include=[np.number]).columns
172