title: What do self-supervised speech models know about Dutch? Analyzing
    advantages of language-specific pre-training
summary: How language-specific are speech representations learned by self-supervised
  models? Existing work has shown that a range of linguistic features can be
  successfully decoded from end-to-end models trained only on speech recordings.
  However, it's less clear to what extent pre-training on specific languages
  improves language-specific linguistic information. Here we test the encoding of
  Dutch phonetic and lexical information in internal representations of
  self-supervised Wav2Vec2 models. Pre-training exclusively on Dutch improves the
  representation of Dutch linguistic features as compared to pre-training on
  similar amounts of English or larger amounts of multilingual data. This
  language-specific advantage is well-detected by trained clustering or
  classification probes, and partially observable using zero-shot metrics.
  Furthermore, the language-specific benefit on linguistic feature encoding
  aligns with downstream performance on Automatic Speech Recognition.
paper_authors._id: 6842d09f619c27947a4a76cd
paper_authors.name: Marianne de Heer Kloots
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-07T05:48:31.333Z
paper_authors._id: 63da7c56e697e5898cbba61c
paper_authors.avatarUrl: /avatars/2385d83f188644f65f7f474c35ae72cc.svg
paper_authors.fullname: Marianne de Heer Kloots
paper_authors.type: user
paper_authors.user: mariannedhk
paper_authors._id: 6842d09f619c27947a4a76ce
paper_authors.name: Hosein Mohebbi
paper_authors._id: 6842d09f619c27947a4a76cf
paper_authors.name: Charlotte Pouw
paper_authors._id: 6842d09f619c27947a4a76d0
paper_authors.name: Gaofei Shen
paper_authors._id: 6842d09f619c27947a4a76d1
paper_authors.name: Willem Zuidema
paper_authors._id: 6842d09f619c27947a4a76d2
paper_authors.name: Martijn Bentum
paper_title: What do self-supervised speech models know about Dutch? Analyzing
    advantages of language-specific pre-training
paper_summary: How language-specific are speech representations learned by self-supervised
  models? Existing work has shown that a range of linguistic features can be
  successfully decoded from end-to-end models trained only on speech recordings.
  However, it's less clear to what extent pre-training on specific languages
  improves language-specific linguistic information. Here we test the encoding of
  Dutch phonetic and lexical information in internal representations of
  self-supervised Wav2Vec2 models. Pre-training exclusively on Dutch improves the
  representation of Dutch linguistic features as compared to pre-training on
  similar amounts of English or larger amounts of multilingual data. This
  language-specific advantage is well-detected by trained clustering or
  classification probes, and partially observable using zero-shot metrics.
  Furthermore, the language-specific benefit on linguistic feature encoding
  aligns with downstream performance on Automatic Speech Recognition.
paper_ai_summary: Self-supervised Wav2Vec2 models encode Dutch linguistic features more accurately when pre-trained exclusively on Dutch data, compared to similar amounts of English or multilingual data, as shown by clustering and classification probes, and demonstrated through improved Automatic Speech Recognition performance.
paper_ai_keywords: Wav2Vec2 phonetic information lexical information self-supervised models clustering probes classification probes Automatic Speech Recognition