title: On the Acquisition of Shared Grammatical Representations in Bilingual
    Language Models
summary: While crosslingual transfer is crucial to contemporary language models'
  multilingual capabilities, how it occurs is not well understood. In this paper,
  we ask what happens to a monolingual language model when it begins to be
  trained on a second language. Specifically, we train small bilingual models for
  which we control the amount of data for each language and the order of language
  exposure. To find evidence of shared multilingual representations, we turn to
  structural priming, a method used to study grammatical representations in
  humans. We first replicate previous crosslingual structural priming results and
  find that after controlling for training data quantity and language exposure,
  there are asymmetrical effects across language pairs and directions. We argue
  that this asymmetry may shape hypotheses about human structural priming
  effects. We also find that structural priming effects are less robust for less
  similar language pairs, highlighting potential limitations of crosslingual
  transfer learning and shared representations for typologically diverse
  languages.
paper_authors._id: 67cb1c41547e3ec05ee92a2b
paper_authors.name: Catherine Arnett
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-03-07T16:18:10.392Z
paper_authors._id: 6384d424ac61472e5e92d403
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6384d424ac61472e5e92d403/RfEyLD3FmEobLcpNE9pNu.png
paper_authors.fullname: Catherine Arnett
paper_authors.type: user
paper_authors.user: catherinearnett
paper_authors._id: 67cb1c41547e3ec05ee92a2c
paper_authors.name: Tyler A. Chang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-03-07T22:57:31.110Z
paper_authors._id: 61830b0b9ad391f401703fc5
paper_authors.avatarUrl: /avatars/0b5623b7db62ca8ff355ba90005ca088.svg
paper_authors.fullname: Tyler Chang
paper_authors.type: user
paper_authors.user: tylerachang
paper_authors._id: 67cb1c41547e3ec05ee92a2d
paper_authors.name: James A. Michaelov
paper_authors._id: 67cb1c41547e3ec05ee92a2e
paper_authors.name: Benjamin K. Bergen
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-03-07T16:18:10.392Z
paper_authors._id: 656914264b8d35447adc5314
paper_authors.avatarUrl: /avatars/04bcaf16a045cdc28e7503ecd5e7311d.svg
paper_authors.fullname: Ben Bergen
paper_authors.type: user
paper_authors.user: bkbergen
paper_title: On the Acquisition of Shared Grammatical Representations in Bilingual
    Language Models
paper_summary: While crosslingual transfer is crucial to contemporary language models'
  multilingual capabilities, how it occurs is not well understood. In this paper,
  we ask what happens to a monolingual language model when it begins to be
  trained on a second language. Specifically, we train small bilingual models for
  which we control the amount of data for each language and the order of language
  exposure. To find evidence of shared multilingual representations, we turn to
  structural priming, a method used to study grammatical representations in
  humans. We first replicate previous crosslingual structural priming results and
  find that after controlling for training data quantity and language exposure,
  there are asymmetrical effects across language pairs and directions. We argue
  that this asymmetry may shape hypotheses about human structural priming
  effects. We also find that structural priming effects are less robust for less
  similar language pairs, highlighting potential limitations of crosslingual
  transfer learning and shared representations for typologically diverse
  languages.
paper_ai_keywords: crosslingual transfer multilingual capabilities monolingual language model bilingual models training data quantity language exposure structural priming grammatical representations asymmetrical effects language pairs directions typologically diverse languages crosslingual transfer learning shared representations