Transformers, Contextualism, and Polysemy
<|reference_start|>Transformers, Contextualism, and Polysemy: The transformer architecture, introduced by Vaswani et al. (2017), is at the heart of the remarkable recent progress in the development of language models, including widely-used chatbots such as Chat-GPT and Claude. In this paper, I argue that we can extract from the way the transformer architecture works a theory of the relationship between context and meaning. I call this the transformer theory, and I argue that it is novel with regard to two related philosophical debates: the contextualism debate regarding the extent of context-sensitivity across natural language, and the polysemy debate regarding how polysemy should be captured within an account of word meaning.<|reference_end|>
@article{grindrod2024transformers,,
    title={Transformers, Contextualism, and Polysemy},
    author={Jumbly Grindrod},
    journal={arXiv preprint arXiv:2404.09577},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.09577},
    primaryClass={cs.CL cs.AI}
}