przvl commited on
Commit
c8fe128
·
verified ·
1 Parent(s): 6edbff0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -18
README.md CHANGED
@@ -117,24 +117,23 @@ Predicted class: Populist (Confidence: 0.90)
117
 
118
  ## Evaluation
119
 
120
- ### Test Set Results (Threshold = 0.5)
121
-
122
- | Metric | Score |
123
- | --------- | ------ |
124
- | Accuracy | 75.99% |
125
- | Precision | 73.78% |
126
- | Recall | 80.66% |
127
- | F1 Score | 77.07% |
128
- | Loss | 0.4959 |
129
-
130
- ### Test Set Results (Optimized Threshold = 0.56)
131
-
132
- | Metric | Score |
133
- | --------- | ------ |
134
- | Accuracy | 76.00% |
135
- | Precision | 76.00% |
136
- | Recall | 76.00% |
137
- | F1 Score | 76.00% |
138
 
139
  ## Limitations
140
 
 
117
 
118
  ## Evaluation
119
 
120
+ For transparency, we compare this model with its smaller variant ([PopEuroBERT-210m](https://huggingface.co/przvl/PopEuroBERT-binary-210m)), both trained and evaluated on the same dataset and splits.
121
+
122
+ ### Test Set Performance (Threshold = 0.5)
123
+
124
+ | Model | Accuracy | Precision | Recall | F1 Score | Loss |
125
+ |--------------------|----------|-----------|--------|----------|--------|
126
+ | **210M** | 75.99% | 73.78% | 80.66% | 77.07% | 0.4959 |
127
+ | **610M (this)** | 80.26% | 78.42% | 83.50% | 80.89% | 0.4631 |
128
+
129
+ ### Test Set Performance (Optimized Threshold)
130
+
131
+ | Model | Threshold | Accuracy | Precision | Recall | F1 Score |
132
+ |--------------------|-----------|----------|-----------|--------|----------|
133
+ | **210M** | 0.56 | 76.00% | 76.00% | 76.00% | 76.00% |
134
+ | **610M (this)** | 0.43 | 79.81% | 76.63% | 85.78% | 80.94% |
135
+
136
+ PopEuroBERT-610m consistently outperforms the 210m variant across all metrics. It especially improves recall and F1 score, suggesting better identification of populist speech. The decision threshold (0.43) was tuned for balanced performance.
 
137
 
138
  ## Limitations
139