On the Acquisition of Shared Grammatical Representations in Bilingual
  Language Models
While crosslingual transfer is crucial to contemporary language models'
multilingual capabilities, how it occurs is not well understood. In this paper,
we ask what happens to a monolingual language model when it begins to be
trained on a second language. Specifically, we train small bilingual models for
which we control the amount of data for each language and the order of language
exposure. To find evidence of shared multilingual representations, we turn to
structural priming, a method used to study grammatical representations in
humans. We first replicate previous crosslingual structural priming results and
find that after controlling for training data quantity and language exposure,
there are asymmetrical effects across language pairs and directions. We argue
that this asymmetry may shape hypotheses about human structural priming
effects. We also find that structural priming effects are less robust for less
similar language pairs, highlighting potential limitations of crosslingual
transfer learning and shared representations for typologically diverse
languages.
67cb1c41547e3ec05ee92a2b
Catherine Arnett
extracted_confirmed
2025-07-15T16:47:15.567Z
6384d424ac61472e5e92d403
https://cdn-avatars.huggingface.co/v1/production/uploads/6384d424ac61472e5e92d403/RfEyLD3FmEobLcpNE9pNu.png
Catherine Arnett
user
catherinearnett
67cb1c41547e3ec05ee92a2c
Tyler A. Chang
extracted_confirmed
2025-03-07T22:57:31.110Z
61830b0b9ad391f401703fc5
https://cdn-avatars.huggingface.co/v1/production/uploads/61830b0b9ad391f401703fc5/oNhyQgbFfsFipmbjvCtYk.jpeg
Tyler Chang
user
tylerachang
67cb1c41547e3ec05ee92a2d
James A. Michaelov
67cb1c41547e3ec05ee92a2e
Benjamin K. Bergen
extracted_confirmed
2025-04-04T22:26:47.484Z
656914264b8d35447adc5314
/avatars/04bcaf16a045cdc28e7503ecd5e7311d.svg
Ben Bergen
user
bkbergen
On the Acquisition of Shared Grammatical Representations in Bilingual
  Language Models
While crosslingual transfer is crucial to contemporary language models'
multilingual capabilities, how it occurs is not well understood. In this paper,
we ask what happens to a monolingual language model when it begins to be
trained on a second language. Specifically, we train small bilingual models for
which we control the amount of data for each language and the order of language
exposure. To find evidence of shared multilingual representations, we turn to
structural priming, a method used to study grammatical representations in
humans. We first replicate previous crosslingual structural priming results and
find that after controlling for training data quantity and language exposure,
there are asymmetrical effects across language pairs and directions. We argue
that this asymmetry may shape hypotheses about human structural priming
effects. We also find that structural priming effects are less robust for less
similar language pairs, highlighting potential limitations of crosslingual
transfer learning and shared representations for typologically diverse
languages.
Research into structural priming in bilingual models reveals asymmetrical effects and limitations in crosslingual transfer learning, particularly between typologically diverse languages.
crosslingual transfer monolingual language model bilingual models structural priming language pairs language exposure typologically diverse languages