We investigate the capability of Transformer large language models (LLMs) to generalize on unseen symbols when trained on  tasks that rely on abstract symbols (e.g.,  variables in programming and mathematics). Such a 'variable-binding' capability  has long been studied in the neuroscience literature as one of the most basic 'reasoning' capabilities. For (i) binary classification tasks, we prove that Transformers can generalize to unseen symbols but require astonishingly large training data. For (ii) tasks with labels dependent on input symbols, we show an ''inverse scaling law'': Transformers fail to generalize to unseen symbols as their embedding dimension increases. For both cases (i) and (ii), we propose a Transformer modification, adding two trainable parameters per head that can reduce the amount of data needed.
When can transformers reason with abstract symbols?
Enric Boix-Adserà Omid Saremi Emmanuel Abbe Samy Bengio Etai Littwin Joshua M. Susskind