3599EN

ρ
MN
0.33 (p=0.00)
FN
0.27 (p=0.00)
MNN 0.30 (p=0.00)
FNN
0.37 (p=0.00)
Avg.
0.32 (p=0.00)

EN

ρ
MN
0.45 (p=0.00)
FN
0.30 (p=0.00)
MNN 0.30 (p=0.00)
FNN
0.40 (p=0.00)
Avg.
0.36 (p=0.00)

EN

ρ
MN
0.01 (p=0.89)
FN
-0.12 (p=0.13)
MNN -0.10 (p=0.22)
FNN
-0.07 (p=0.41)
Avg.
-0.07 (p=0.07)

mBERT
ES

0.23 (p=0.01)
0.07 (p=0.42)
0.16 (p=0.03)
0.16 (p=0.06)
0.16 (p=0.00)

XLM-R
ES

0.46 (p=0.00)
0.35 (p=0.00)
0.38 (p=0.00)
0.48 (p=0.00)
0.41 (p=0.00)

mT5

ES

0.14 (p=0.08)
0.13 (p=0.12)
0.12 (p=0.11)
0.28 (p=0.00)
0.17 (p=0.00)

DE

-0.14 (p=0.09)
-0.01 (p=0.89)
-0.10 (p=0.23)
0.03 (p=0.69)
-0.05 (p=0.21)

FR

0.10 (p=0.21)
0.14 (p=0.08)
0.08 (p=0.32)
0.08 (p=0.30)
0.10 (p=0.01)

DE

0.35 (p=0.00)
0.45 (p=0.00)
0.22 (p=0.01)
0.11 (p=0.16)
0.28 (p=0.00)

FR

0.48 (p=0.00)
0.33 (p=0.00)
0.32 (p=0.00)
0.36 (p=0.00)
0.37 (p=0.00)

DE

0.14 (p=0.08)
0.00 (p=0.99)
0.03 (p=0.74)
0.04 (p=0.58)
0.05 (p=0.23)

FR

0.25 (p=0.00)
0.14 (p=0.08)
0.11 (p=0.18)
0.11 (p=0.16)
0.15 (p=0.00)
Are Pretrained Multilingual Models Equally Fair Across Languages?