• IR→O, which maps pairs of inputs and ratio-
nales to outputs.
• I→O, which maps inputs to outputs.
We provide input-output formatting in Table 8
(Appendix A.3). Using these building blocks, we
can instantiate two important approaches.
Pipeline Model (I→R;R→O) This architecture
composes I→R with R→O, each of which is
trained entirely separately, for a total of 440M pa-
rameters. It is illustrated in Figure 2 and is faithful-
by-construction (with caveats; see Jacovi and Gold-
berg, 2021). The vast majority of prior work using
pipelines has focused only on extractive rationales
(see Table 2).
Self-Rationalizing Model (I→OR) A joint, self-
rationalizing model (Melis and Jaakkola, 2018),
illustrated in Figure 3, predicts both a label and
rationale. This is the most common approach to
free-text rationalization (Hendricks et al., 2016;
Kim et al., 2018; Hancock et al., 2018; Camburu
et al., 2018; Ehsan et al., 2018; Liu et al., 2019a;
Wu and Mooney, 2019; Narang et al., 2020; Do
Measuring Association Between Labels and Free-Text Rationales