title: Strong Membership Inference Attacks on Massive Datasets and (Moderately)
    Large Language Models
summary: State-of-the-art membership inference attacks (MIAs) typically require
  training many reference models, making it difficult to scale these attacks to
  large pre-trained language models (LLMs). As a result, prior research has
  either relied on weaker attacks that avoid training reference models (e.g.,
  fine-tuning attacks), or on stronger attacks applied to small-scale models and
  datasets. However, weaker attacks have been shown to be brittle - achieving
  close-to-arbitrary success - and insights from strong attacks in simplified
  settings do not translate to today's LLMs. These challenges have prompted an
  important question: are the limitations observed in prior work due to attack
  design choices, or are MIAs fundamentally ineffective on LLMs? We address this
  question by scaling LiRA - one of the strongest MIAs - to GPT-2 architectures
  ranging from 10M to 1B parameters, training reference models on over 20B tokens
  from the C4 dataset. Our results advance the understanding of MIAs on LLMs in
  three key ways: (1) strong MIAs can succeed on pre-trained LLMs; (2) their
  effectiveness, however, remains limited (e.g., AUC<0.7) in practical settings;
  and, (3) the relationship between MIA success and related privacy metrics is
  not as straightforward as prior work has suggested.
paper_authors._id: 6835727f9da2b91fb4e30473
paper_authors.name: Jamie Hayes
paper_authors._id: 6835727f9da2b91fb4e30474
paper_authors.name: Ilia Shumailov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:44:17.787Z
paper_authors._id: 6475c2794766357252e69e9f
paper_authors.avatarUrl: /avatars/757ed789423113369868e972f21ce559.svg
paper_authors.fullname: i
paper_authors.type: user
paper_authors.user: iliashum
paper_authors._id: 6835727f9da2b91fb4e30475
paper_authors.name: Christopher A. Choquette-Choo
paper_authors._id: 6835727f9da2b91fb4e30476
paper_authors.name: Matthew Jagielski
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:44:27.202Z
paper_authors._id: 654a4a3b6a46bad8c0f2ab80
paper_authors.avatarUrl: /avatars/08caf74525ea3e46fe58c20eb1f3d64d.svg
paper_authors.fullname: Matthew Jagielski
paper_authors.type: user
paper_authors.user: jagielski
paper_authors._id: 6835727f9da2b91fb4e30477
paper_authors.name: George Kaissis
paper_authors._id: 6835727f9da2b91fb4e30478
paper_authors.name: Katherine Lee
paper_authors._id: 6835727f9da2b91fb4e30479
paper_authors.name: Milad Nasr
paper_authors._id: 6835727f9da2b91fb4e3047a
paper_authors.name: Sahra Ghalebikesabi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:44:52.007Z
paper_authors._id: 633a01c6c0b7fd23493cb934
paper_authors.avatarUrl: /avatars/8592c548b2f8324c43c5fed2599e6074.svg
paper_authors.fullname: Sahra Ghalebikesabi
paper_authors.type: user
paper_authors.user: sahra
paper_authors._id: 6835727f9da2b91fb4e3047b
paper_authors.name: Niloofar Mireshghallah
paper_authors._id: 6835727f9da2b91fb4e3047c
paper_authors.name: Meenatchi Sundaram Mutu Selva Annamalai
paper_authors._id: 6835727f9da2b91fb4e3047d
paper_authors.name: Igor Shilov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:45:06.168Z
paper_authors._id: 6579c19a7a0f9b8a1abdccf1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/vhsOT4mneeJX0KtFi-uDb.jpeg
paper_authors.fullname: Igor Shilov
paper_authors.type: user
paper_authors.user: ffuuugor
paper_authors._id: 6835727f9da2b91fb4e3047e
paper_authors.name: Matthieu Meeus
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T09:00:24.642Z
paper_authors._id: 63d66b586f49aa82305e848d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63d66b586f49aa82305e848d/ZlN9lKBwSEsEKl6yqddZZ.jpeg
paper_authors.fullname: Matthieu Meeus
paper_authors.type: user
paper_authors.user: matthieumeeus97
paper_authors._id: 6835727f9da2b91fb4e3047f
paper_authors.name: Yves-Alexandre de Montjoye
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T12:45:18.016Z
paper_authors._id: 6694f6b3284bcf975861b818
paper_authors.avatarUrl: /avatars/37cd86306142027663b6471b0033ef58.svg
paper_authors.fullname: Yves-Alexandre de Montjoye
paper_authors.type: user
paper_authors.user: yvesalexandre
paper_authors._id: 6835727f9da2b91fb4e30480
paper_authors.name: Franziska Boenisch
paper_authors._id: 6835727f9da2b91fb4e30481
paper_authors.name: Adam Dziedzic
paper_authors._id: 6835727f9da2b91fb4e30482
paper_authors.name: A. Feder Cooper
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-27T20:00:14.988Z
paper_authors._id: 663c3b587e7bc3d3e4a54ffb
paper_authors.avatarUrl: /avatars/681abf7e4a85184667015cefefa226c6.svg
paper_authors.fullname: A. Feder Cooper
paper_authors.type: user
paper_authors.user: pasta41
paper_title: Strong Membership Inference Attacks on Massive Datasets and (Moderately)
    Large Language Models
paper_summary: State-of-the-art membership inference attacks (MIAs) typically require
  training many reference models, making it difficult to scale these attacks to
  large pre-trained language models (LLMs). As a result, prior research has
  either relied on weaker attacks that avoid training reference models (e.g.,
  fine-tuning attacks), or on stronger attacks applied to small-scale models and
  datasets. However, weaker attacks have been shown to be brittle - achieving
  close-to-arbitrary success - and insights from strong attacks in simplified
  settings do not translate to today's LLMs. These challenges have prompted an
  important question: are the limitations observed in prior work due to attack
  design choices, or are MIAs fundamentally ineffective on LLMs? We address this
  question by scaling LiRA - one of the strongest MIAs - to GPT-2 architectures
  ranging from 10M to 1B parameters, training reference models on over 20B tokens
  from the C4 dataset. Our results advance the understanding of MIAs on LLMs in
  three key ways: (1) strong MIAs can succeed on pre-trained LLMs; (2) their
  effectiveness, however, remains limited (e.g., AUC<0.7) in practical settings;
  and, (3) the relationship between MIA success and related privacy metrics is
  not as straightforward as prior work has suggested.
paper_ai_summary: Scaling LiRA membership inference attacks to large pre-trained language models shows that while these attacks can succeed, their effectiveness is limited and does not definitively correlate with privacy metrics.
paper_ai_keywords: membership inference attacks MIAs reference models fine-tuning attacks pre-trained language models LLMs LiRA GPT-2 tokens C4 dataset AUC privacy metrics