Random Majority-Vote
Accuracy
Accuracy
33.39
33.33
33.75
33.33
20.0
20.31

∆
–0.14 (–0.05‡)
–4.32
–6.14
Table 9: Label accuracy on baseline I→O T5 models versus their rationalizing I→OR variants fine-tuned for each
dataset. We observe that adding rationalization results in some loss in accuracy. We also validate that T5-Base
models outperform other architectures. Source of prior results in parentheses: † Narang et al. (2020) using T5, ‡
Camburu et al. (2018) using a bi-directional LSTM, and § Rajani et al. (2019) using BERT.

I→OR
90.81 (83.96‡, 90.9†)
64.84
55.61 (59.4†)

I→O (T5)
90.95 (84.01‡)
69.16 (63.8§)
61.75

Dataset
E-SNLI
CoS-E v1.0
CoS-E v1.11

I→OR
90.81 (83.96‡)
64.84
55.61

I→R;R→O
89.11 (81.71‡)
53.47
45.45

∆
–1.70 (–2.25‡)
–11.37
–10.16
Measuring Association Between Labels and Free-Text Rationales