---
license: mit
datasets:
  - ai4privacy/open-pii-masking-500k-ai4privacy
language:
  - fr
  - en
  - de
  - te
  - hi
  - it
  - es
  - nl
base_model:
  - answerdotai/ModernBERT-base
library_name: transformers
tags:
  - PII
  - redaction
  - anonymisation
  - token-classification
model-index:
  - name: multilingual-anonymiser-openpii-ai4privacy
    results:
      - task:
          type: token-classification
          name: PII Masking and Classification
        dataset:
          type: ai4privacy/open-pii-masking-500k-ai4privacy
          name: Open PII Masking 500K
          split: test
        metrics:
          - type: f1
            value: 0.9150
            name: F1 Score
          - type: precision
            value: 0.8761
            name: Precision
          - type: recall
            value: 0.9576
            name: Recall
          - type: accuracy
            value: 0.9503
            name: Accuracy
---

# Multilingual Anonymiser OpenPII (Ai4Privacy)

This model is designed to **redact and classify Personally Identifiable Information (PII)** from multilingual text. It has been fine-tuned on the [open-pii-masking-500k-ai4privacy](https://huggingface.co/datasets/ai4privacy/open-pii-masking-500k-ai4privacy) dataset and supports multiple languages, including French (fr), English (en), German (de), Telugu (te), Hindi (hi), Italian (it), Spanish (es), and Dutch (nl).

---

## Evaluation Metrics

The table below summarizes the detailed evaluation results per PII label. Metrics are presented as percentages rounded to two decimal places. For the "O" (Non-PII) label, precision, recall, and F1 score are not applicable (n/a) due to the absence of true positives.

| **Label**          | **TP** | **FP** | **FN** | **Accuracy** | **Precision** | **Recall** | **F1 Score** |
|--------------------|:------:|:------:|:------|:------------:|:-------------:|:----------:|:------------:|
| O (Non-PII)        | 0      | 734    | 0      | 98.97%       | n/a           | n/a        | n/a          |
| GIVENNAME          | 6623   | 661    | 352    | 86.73%       | 90.93%        | 94.95%     | 92.90%       |
| SURNAME            | 2786   | 877    | 162    | 72.84%       | 76.06%        | 94.50%     | 84.28%       |
| CITY               | 1763   | 216    | 225    | 79.99%       | 89.09%        | 88.68%     | 88.88%       |
| DATE               | 2195   | 1      | 3      | 99.82%       | 99.95%        | 99.86%     | 99.91%       |
| AGE                | 176    | 7      | 2      | 95.14%       | 96.17%        | 98.88%     | 97.51%       |
| EMAIL              | 2981   | 0      | 0      | 100.0%       | 100.0%        | 100.0%     | 100.0%       |
| CREDITCARDNUMBER   | 601    | 57     | 35     | 86.72%       | 91.34%        | 94.50%     | 92.89%       |
| SEX                | 103    | 45     | 1      | 69.13%       | 69.59%        | 99.04%     | 81.75%       |
| SOCIALNUM          | 364    | 134    | 20     | 70.27%       | 73.09%        | 94.79%     | 82.54%       |
| TIME               | 1631   | 1      | 3      | 99.76%       | 99.94%        | 99.82%     | 99.88%       |
| TELEPHONENUM       | 3537   | 10     | 9      | 99.47%       | 99.72%        | 99.75%     | 99.73%       |
| IDCARDNUM          | 1540   | 314    | 148    | 76.92%       | 83.06%        | 91.23%     | 86.96%       |
| ZIPCODE            | 311    | 39     | 16     | 84.97%       | 88.86%        | 95.11%     | 91.87%       |
| DRIVERLICENSENUM   | 296    | 143    | 26     | 63.66%       | 67.43%        | 91.93%     | 77.79%       |
| PASSPORTNUM        | 482    | 285    | 25     | 60.86%       | 62.84%        | 95.07%     | 75.67%       |
| TITLE              | 224    | 68     | 78     | 60.54%       | 76.71%        | 74.17%     | 75.42%       |
| BUILDINGNUM        | 292    | 45     | 14     | 83.19%       | 86.65%        | 95.42%     | 90.85%       |
| STREET             | 1272   | 155    | 67     | 85.14%       | 89.14%        | 94.99%     | 91.97%       |
| TAXNUM             | 471    | 101    | 34     | 77.72%       | 82.34%        | 93.27%     | 87.47%       |
| GENDER             | 123    | 35     | 9      | 73.65%       | 77.85%        | 93.18%     | 84.83%       |

### Overall Evaluation
- **Accuracy:** 95.03%  
- **Precision:** 87.61%  
- **Recall:** 95.76%  
- **F1 Score:** 91.50%  

- **Total True Positives (TP):** 27,771  
- **Total False Positives (FP):** 3,928  
- **Total False Negatives (FN):** 1,229  

### Macro-Averaged Metrics
- **Accuracy:** 82.17%  
- **Precision:** 80.99%  
- **Recall:** 89.96%  
- **F1 Score:** 84.91%  

---

## Model Behavior & Limitations

- **Evaluation Focus:**  
  The metrics above reflect performance on the test split of the [open-pii-masking-500k-ai4privacy](https://huggingface.co/datasets/ai4privacy/open-pii-masking-500k-ai4privacy) dataset. This model both redacts and classifies PII into specific categories (e.g., GIVENNAME, EMAIL). Real-world performance may vary depending on text domain and language, so additional validation is recommended. For support, contact **support@ai4privacy.com**.

- **Strengths:**  
  - High recall (95.76%) ensures most PII is detected.  
  - Exceptional performance on labels like "EMAIL" (100% F1), "DATE" (99.91% F1), and "TIME" (99.88% F1).  

- **Limitations:**  
  - Lower precision for labels such as "PASSPORTNUM" (62.84%) and "DRIVERLICENSENUM" (67.43%), indicating a higher rate of false positives.  
  - The "O" (Non-PII) label has no true positives, making precision, recall, and F1 score not applicable (n/a).  

---

## Disclaimer

This model card details the evaluation metrics and fine-tuning parameters for the multilingual anonymiser with PII classification capabilities. **Please note:**  
- The model is provided **as-is** under the MIT License.  
- It is intended for both redaction and PII classification purposes.  
- Users should thoroughly test and evaluate its performance on their specific datasets before deploying in production environments.

---

*Ai4Privacy – Committed to protecting personal data in the age of AI.*

---