MultiNLI
Matched
TinyBERT
Natural Language Inference
82.5
MultiNLI
Mismatched
TinyBERT
Natural Language Inference
81.8
Quora Question Pairs
Accuracy
TinyBERT
Paraphrase Identification
71.3
SST-2 Binary classification
Accuracy
TinyBERT
Sentiment Analysis
92.6
QNLI
Accuracy
TinyBERT
Natural Language Inference
87.7
MRPC
Accuracy
TinyBERT
Semantic Textual Similarity
86.4
RTE
Accuracy
TinyBERT
Natural Language Inference
62.9
CoLA
Accuracy
TinyBERT
Linguistic Acceptability
43.3
STS Benchmark
Pearson Correlation
TinyBERT
Semantic Textual Similarity
0.799
MultiNLI Dev
Matched
TinyBERT (M=6;d'=768;d'i=3072)
Natural Language Inference
84.5
MultiNLI Dev
Mismatched
TinyBERT (M=6;d'=768;d'i=3072)
Natural Language Inference
84.5
MRPC Dev
Accuracy
TinyBERT (M=6;d'=768;d'i=3072)
Semantic Textual Similarity
86.3
CoLA Dev
Accuracy
TinyBERT (M=6;d' =768;d'i=3072)
Linguistic Acceptability
54
SQuAD1.1 dev
EM
TinyBERT (M=6;d' =768;d'i=3072)
Question Answering
79.7
SQuAD1.1 dev
F1
TinyBERT (M=6;d' =768;d'i=3072)
Question Answering
87.5
SQuAD2.0 dev
EM
TinyBERT (M=6;d' =768;d'i=3072)
Question Answering
69.9
SQuAD2.0 dev
F1
TinyBERT (M=6;d' =768;d'i=3072)
Question Answering
73.4