Experiment 3: Generalization across question formats. A main challenge
in constructing natural language interfaces to discrete reasoners like a calculator is in
handling language variability. One of the most appealing characteristics of language
models is their ability to abstract away from such variability. Here we test the ability
of the model to generalize over formats of arithmetic problems. Experiments are for
single-operation problems.
MRKL Systems