Update README.md
Browse files
README.md
CHANGED
@@ -117,24 +117,23 @@ Predicted class: Populist (Confidence: 0.90)
|
|
117 |
|
118 |
## Evaluation
|
119 |
|
120 |
-
|
121 |
-
|
122 |
-
|
123 |
-
|
124 |
-
| Accuracy
|
125 |
-
|
126 |
-
|
|
127 |
-
|
|
128 |
-
|
129 |
-
|
130 |
-
|
131 |
-
|
132 |
-
|
133 |
-
|
|
134 |
-
|
|
135 |
-
|
136 |
-
|
137 |
-
| F1 Score | 76.00% |
|
138 |
|
139 |
## Limitations
|
140 |
|
|
|
117 |
|
118 |
## Evaluation
|
119 |
|
120 |
+
For transparency, we compare this model with its smaller variant ([PopEuroBERT-210m](https://huggingface.co/przvl/PopEuroBERT-binary-210m)), both trained and evaluated on the same dataset and splits.
|
121 |
+
|
122 |
+
### Test Set Performance (Threshold = 0.5)
|
123 |
+
|
124 |
+
| Model | Accuracy | Precision | Recall | F1 Score | Loss |
|
125 |
+
|--------------------|----------|-----------|--------|----------|--------|
|
126 |
+
| **210M** | 75.99% | 73.78% | 80.66% | 77.07% | 0.4959 |
|
127 |
+
| **610M (this)** | 80.26% | 78.42% | 83.50% | 80.89% | 0.4631 |
|
128 |
+
|
129 |
+
### Test Set Performance (Optimized Threshold)
|
130 |
+
|
131 |
+
| Model | Threshold | Accuracy | Precision | Recall | F1 Score |
|
132 |
+
|--------------------|-----------|----------|-----------|--------|----------|
|
133 |
+
| **210M** | 0.56 | 76.00% | 76.00% | 76.00% | 76.00% |
|
134 |
+
| **610M (this)** | 0.43 | 79.81% | 76.63% | 85.78% | 80.94% |
|
135 |
+
|
136 |
+
PopEuroBERT-610m consistently outperforms the 210m variant across all metrics. It especially improves recall and F1 score, suggesting better identification of populist speech. The decision threshold (0.43) was tuned for balanced performance.
|
|
|
137 |
|
138 |
## Limitations
|
139 |
|