title: Pixtral 12B
summary: We introduce Pixtral-12B, a 12--billion-parameter multimodal language model.
  Pixtral-12B is trained to understand both natural images and documents,
  achieving leading performance on various multimodal benchmarks, surpassing a
  number of larger models. Unlike many open-source models, Pixtral is also a
  cutting-edge text model for its size, and does not compromise on natural
  language performance to excel in multimodal tasks. Pixtral uses a new vision
  encoder trained from scratch, which allows it to ingest images at their natural
  resolution and aspect ratio. This gives users flexibility on the number of
  tokens used to process an image. Pixtral is also able to process any number of
  images in its long context window of 128K tokens. Pixtral 12B substanially
  outperforms other open models of similar sizes (Llama-3.2 11B \& Qwen-2-VL 7B).
  It also outperforms much larger open models like Llama-3.2 90B while being 7x
  smaller. We further contribute an open-source benchmark, MM-MT-Bench, for
  evaluating vision-language models in practical scenarios, and provide detailed
  analysis and code for standardized evaluation protocols for multimodal LLMs.
  Pixtral-12B is released under Apache 2.0 license.
paper_authors._id: 670746ee3e510db7639040f2
paper_authors.name: Pravesh Agrawal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:44:28.232Z
paper_authors._id: 66dfa475a36a3baebd55266b
paper_authors.avatarUrl: /avatars/2c46210d159c200d4db9160fc5ecfe57.svg
paper_authors.fullname: Pravesh Agrawal
paper_authors.type: user
paper_authors.user: pragra
paper_authors._id: 670746ee3e510db7639040f3
paper_authors.name: Szymon Antoniak
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:44:33.689Z
paper_authors._id: 63f767c4bd28622c9b984637
paper_authors.avatarUrl: /avatars/126a23ca5b3adc89418b7a53b21e50e6.svg
paper_authors.fullname: Szymon Antoniak
paper_authors.type: user
paper_authors.user: Simontwice
paper_authors._id: 670746ee3e510db7639040f4
paper_authors.name: Emma Bou Hanna
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:44:39.014Z
paper_authors._id: 655dc9138e56f09d55197cac
paper_authors.avatarUrl: /avatars/764edb5c5e1003eb71cf6f1bea563f4b.svg
paper_authors.fullname: Emma Bou Hanna
paper_authors.type: user
paper_authors.user: EmmaBH
paper_authors._id: 670746ee3e510db7639040f5
paper_authors.name: Devendra Chaplot
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:44:44.258Z
paper_authors._id: 65143e1c4f08b815c8db57a0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65143e1c4f08b815c8db57a0/JqkwKiJmLFRkH0NK3L8XH.jpeg
paper_authors.fullname: Devendra Singh Chaplot
paper_authors.type: user
paper_authors.user: devendrachaplot
paper_authors._id: 670746ee3e510db7639040f6
paper_authors.name: Jessica Chudnovsky
paper_authors._id: 670746ee3e510db7639040f7
paper_authors.name: Saurabh Garg
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:45:04.979Z
paper_authors._id: 64a8a87e4a702899835e9b45
paper_authors.avatarUrl: /avatars/b6c60f63337a78a1bf3943885f4d7a18.svg
paper_authors.fullname: Saurabh Garg
paper_authors.type: user
paper_authors.user: saurabhgarg
paper_authors._id: 670746ee3e510db7639040f8
paper_authors.name: Theophile Gervet
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:45:10.366Z
paper_authors._id: 644e822cd6001776ed7700e4
paper_authors.avatarUrl: /avatars/58a72e6bf8ede3cee3de54873231fd3e.svg
paper_authors.fullname: Theophile Gervet
paper_authors.type: user
paper_authors.user: TheophileGervet
paper_authors._id: 670746ee3e510db7639040f9
paper_authors.name: Soham Ghosh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:45:27.823Z
paper_authors._id: 66bbbc31e314ecd9cc24cebc
paper_authors.avatarUrl: /avatars/7f9b596a3a1c7243cea118423b61b97c.svg
paper_authors.fullname: Soham Ghosh
paper_authors.type: user
paper_authors.user: sohamghosh121
paper_authors._id: 670746ee3e510db7639040fa
paper_authors.name: Amélie Héliou
paper_authors._id: 670746ee3e510db7639040fb
paper_authors.name: Paul Jacob
paper_authors._id: 670746ee3e510db7639040fc
paper_authors.name: Albert Q. Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:45:46.987Z
paper_authors._id: 6137dff56ecbbda5c86755e9
paper_authors.avatarUrl: /avatars/1aadefd3acb7ad24df15463dadceb7d4.svg
paper_authors.fullname: Albert Q Jiang
paper_authors.type: user
paper_authors.user: aqj213
paper_authors._id: 670746ee3e510db7639040fd
paper_authors.name: Timothée Lacroix
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:46:00.153Z
paper_authors._id: 6503364ceda77aafd4a68a48
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6503364ceda77aafd4a68a48/POQqon4zWQEpFTPzHS4uu.png
paper_authors.fullname: timothee lacroix
paper_authors.type: user
paper_authors.user: timlacroix
paper_authors._id: 670746ee3e510db7639040fe
paper_authors.name: Guillaume Lample
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:46:05.767Z
paper_authors._id: 64882fc9508d0656d43dd5b3
paper_authors.avatarUrl: /avatars/c3bc3656e4ff05663b54c6dd4f241891.svg
paper_authors.fullname: Guillaume Lample
paper_authors.type: user
paper_authors.user: glample
paper_authors._id: 670746ee3e510db7639040ff
paper_authors.name: Diego Las Casas
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:46:11.532Z
paper_authors._id: 657608c427c0e2fd6c894ced
paper_authors.avatarUrl: /avatars/7f1ca5f75ca03cec69560584496558c9.svg
paper_authors.fullname: Diego de las Casas
paper_authors.type: user
paper_authors.user: diegolascasas
paper_authors._id: 670746ee3e510db763904100
paper_authors.name: Thibaut Lavril
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:46:16.884Z
paper_authors._id: 66c705caf3f9994f2adf8c77
paper_authors.avatarUrl: /avatars/e0322a12703693d10005113cbeed3207.svg
paper_authors.fullname: Thibaut Lavril
paper_authors.type: user
paper_authors.user: thibmistral
paper_authors._id: 670746ee3e510db763904101
paper_authors.name: Teven Le Scao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:46:22.313Z
paper_authors._id: 5e67bed6100906368940747b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1583858339980-5e67bed6100906368940747b.jpeg
paper_authors.fullname: Teven Le Scao
paper_authors.type: user
paper_authors.user: teven
paper_authors._id: 670746ee3e510db763904102
paper_authors.name: Andy Lo
paper_authors._id: 670746ee3e510db763904103
paper_authors.name: William Marshall
paper_authors._id: 670746ee3e510db763904104
paper_authors.name: Louis Martin
paper_authors._id: 670746ee3e510db763904105
paper_authors.name: Arthur Mensch
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:47:16.994Z
paper_authors._id: 645d28525ebf379fd6d9ae41
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/645d28525ebf379fd6d9ae41/qVEKQk4gPlumsZE3ZorWG.jpeg
paper_authors.fullname: Arthur Mensch
paper_authors.type: user
paper_authors.user: arthurmensch
paper_authors._id: 670746ee3e510db763904106
paper_authors.name: Pavankumar Muddireddy
paper_authors._id: 670746ee3e510db763904107
paper_authors.name: Valera Nemychnikova
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:47:32.137Z
paper_authors._id: 665efa6e8947302aa2bfc569
paper_authors.avatarUrl: /avatars/58e044ad6ef6040a06b0fff4945ae988.svg
paper_authors.fullname: Valera Nemychnikova
paper_authors.type: user
paper_authors.user: sooobus
paper_authors._id: 670746ee3e510db763904108
paper_authors.name: Marie Pellat
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:47:39.511Z
paper_authors._id: 62c69cd35aae1c624ca9e42c
paper_authors.avatarUrl: /avatars/13824164766fbb06f7d45d6d38d96407.svg
paper_authors.fullname: Marie Pellat
paper_authors.type: user
paper_authors.user: mpellat
paper_authors._id: 670746ee3e510db763904109
paper_authors.name: Patrick Von Platen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:47:46.655Z
paper_authors._id: 5dfcb1aada6d0311fd3d5448
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg
paper_authors.fullname: Patrick von Platen
paper_authors.type: user
paper_authors.user: patrickvonplaten
paper_authors._id: 670746ee3e510db76390410a
paper_authors.name: Nikhil Raghuraman
paper_authors._id: 670746ee3e510db76390410b
paper_authors.name: Baptiste Rozière
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:48:02.482Z
paper_authors._id: 62d063dac375d0c84255b9a1
paper_authors.avatarUrl: /avatars/de0fc34bad8c761210c0895ebfa4feba.svg
paper_authors.fullname: Baptiste Roziere
paper_authors.type: user
paper_authors.user: broz
paper_authors._id: 670746ee3e510db76390410c
paper_authors.name: Alexandre Sablayrolles
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:48:09.427Z
paper_authors._id: 6391c7456176fbc67b9ecd2c
paper_authors.avatarUrl: /avatars/4aa9be94d8284118b8018362abf11f01.svg
paper_authors.fullname: Alexandre Sablayrolles
paper_authors.type: user
paper_authors.user: alexsablay
paper_authors._id: 670746ee3e510db76390410d
paper_authors.name: Lucile Saulnier
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:48:23.255Z
paper_authors._id: 650cb0ebc705442b280160c4
paper_authors.avatarUrl: /avatars/c5abfcddfa66505085178fa430779ed0.svg
paper_authors.fullname: Lucile Saulnier
paper_authors.type: user
paper_authors.user: LucileSaulnier
paper_authors._id: 670746ee3e510db76390410e
paper_authors.name: Romain Sauvestre
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:48:38.293Z
paper_authors._id: 66467938b7976a1d8d029b12
paper_authors.avatarUrl: /avatars/7c4588aa2f50e67071b8f09cef13b005.svg
paper_authors.fullname: Sauvestre
paper_authors.type: user
paper_authors.user: romainsauvestre
paper_authors._id: 670746ee3e510db76390410f
paper_authors.name: Wendy Shang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:48:44.673Z
paper_authors._id: 663d3aa7bc8177e039c17b3e
paper_authors.avatarUrl: /avatars/b2ba474b94634472611105cf2968d077.svg
paper_authors.fullname: Wendy Shang
paper_authors.type: user
paper_authors.user: wendyshang
paper_authors._id: 670746ee3e510db763904110
paper_authors.name: Roman Soletskyi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:49:05.189Z
paper_authors._id: 6535a3d33da0ff3c70b6b3d2
paper_authors.avatarUrl: /avatars/bc40dea410af9b2bd4881d7004eed9a7.svg
paper_authors.fullname: Roman Soletskyi
paper_authors.type: user
paper_authors.user: romansoletskyi
paper_authors._id: 670746ee3e510db763904111
paper_authors.name: Lawrence Stewart
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:49:19.202Z
paper_authors._id: 664263de23fcdb3879e5bf36
paper_authors.avatarUrl: /avatars/2a0f44b1a84c359c4b24c306c49f9e6a.svg
paper_authors.fullname: Lawrence Stewart
paper_authors.type: user
paper_authors.user: lmms
paper_authors._id: 670746ee3e510db763904112
paper_authors.name: Pierre Stock
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:49:26.789Z
paper_authors._id: 650c993d2751c84306aba92b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650c993d2751c84306aba92b/uS04hveXAu6CEszCy-8_G.jpeg
paper_authors.fullname: Pierre Stock
paper_authors.type: user
paper_authors.user: pstock
paper_authors._id: 670746ee3e510db763904113
paper_authors.name: Joachim Studnia
paper_authors._id: 670746ee3e510db763904114
paper_authors.name: Sandeep Subramanian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:49:37.492Z
paper_authors._id: 627bf27cf19c5eb46d54cea8
paper_authors.avatarUrl: /avatars/b8ca0b4e841858c1d234671187234f56.svg
paper_authors.fullname: Sandeep Subramanian
paper_authors.type: user
paper_authors.user: MaximumEntropy
paper_authors._id: 670746ee3e510db763904115
paper_authors.name: Sagar Vaze
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T08:49:44.074Z
paper_authors._id: 643838e5c5a91b84ece168dd
paper_authors.avatarUrl: /avatars/abebd42399decafbccc8579faa34e7d3.svg
paper_authors.fullname: Sagar Vaze
paper_authors.type: user
paper_authors.user: sgvaze
paper_authors._id: 670746ee3e510db763904116
paper_authors.name: Thomas Wang
paper_title: Pixtral 12B
paper_summary: We introduce Pixtral-12B, a 12--billion-parameter multimodal language model.
  Pixtral-12B is trained to understand both natural images and documents,
  achieving leading performance on various multimodal benchmarks, surpassing a
  number of larger models. Unlike many open-source models, Pixtral is also a
  cutting-edge text model for its size, and does not compromise on natural
  language performance to excel in multimodal tasks. Pixtral uses a new vision
  encoder trained from scratch, which allows it to ingest images at their natural
  resolution and aspect ratio. This gives users flexibility on the number of
  tokens used to process an image. Pixtral is also able to process any number of
  images in its long context window of 128K tokens. Pixtral 12B substanially
  outperforms other open models of similar sizes (Llama-3.2 11B \& Qwen-2-VL 7B).
  It also outperforms much larger open models like Llama-3.2 90B while being 7x
  smaller. We further contribute an open-source benchmark, MM-MT-Bench, for
  evaluating vision-language models in practical scenarios, and provide detailed
  analysis and code for standardized evaluation protocols for multimodal LLMs.
  Pixtral-12B is released under Apache 2.0 license.
paper_ai_keywords: Pixtral-12B multimodal language model multimodal benchmarks vision encoder natural resolution aspect ratio long context window vision-language models MM-MT-Bench standardized evaluation protocols multimodal LLMs