Table 2 shows the accuracy for all combinations of training and testing on digits
and on words. We notice that in-distribution performance (training and testing on
the same type of input) is close to 100 % accuracy. We further observe that training

9

on words generalizes well to digits. Interestingly, the inverse is not true and the per-
formance is much lower, indicating the underlying diﬀerence in the representations
of the numbers.

Test

Train Digits Words
0.156
Digits
0.988
Words

1.0
0.987

Table 2: Robustness to wordings. Accuracy for all combination of training and testing
on numbers written as digits and as words. Results are averaged across all the numbers of
digits.
MRKL Systems