title: ANOLE: An Open, Autoregressive, Native Large Multimodal Models for
    Interleaved Image-Text Generation
summary: Previous open-source large multimodal models (LMMs) have faced several
  limitations: (1) they often lack native integration, requiring adapters to
  align visual representations with pre-trained large language models (LLMs); (2)
  many are restricted to single-modal generation; (3) while some support
  multimodal generation, they rely on separate diffusion models for visual
  modeling and generation. To mitigate these limitations, we present Anole, an
  open, autoregressive, native large multimodal model for interleaved image-text
  generation. We build Anole from Meta AI's Chameleon, adopting an innovative
  fine-tuning strategy that is both data-efficient and parameter-efficient. Anole
  demonstrates high-quality, coherent multimodal generation capabilities. We have
  open-sourced our model, training framework, and instruction tuning data.
paper_authors._id: 668cf5f763d1bc005ba9d481
paper_authors.name: Ethan Chern
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-09T09:18:31.173Z
paper_authors._id: 64bb5f9d8e051085bace4d1e
paper_authors.avatarUrl: /avatars/15ccbb78c6131dfe46b7a9d8e7d1a31f.svg
paper_authors.fullname: Ethan Chern
paper_authors.type: user
paper_authors.user: ethanchern
paper_authors._id: 668cf5f763d1bc005ba9d482
paper_authors.name: Jiadi Su
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-09T09:34:27.601Z
paper_authors._id: 65437c82d3db5d5336af93ff
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65437c82d3db5d5336af93ff/YDfpywk15IigQwcAoMu5n.jpeg
paper_authors.fullname: Jiadi Su
paper_authors.type: user
paper_authors.user: JoyBoy-Su
paper_authors._id: 668cf5f763d1bc005ba9d483
paper_authors.name: Yan Ma
paper_authors._id: 668cf5f763d1bc005ba9d484
paper_authors.name: Pengfei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-09T09:35:09.039Z
paper_authors._id: 6144a0c4ff1146bbd84d9865
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1661715958139-6144a0c4ff1146bbd84d9865.png
paper_authors.fullname: Pengfei Liu
paper_authors.type: user
paper_authors.user: Pengfei
paper_title: ANOLE: An Open, Autoregressive, Native Large Multimodal Models for
    Interleaved Image-Text Generation
paper_summary: Previous open-source large multimodal models (LMMs) have faced several
  limitations: (1) they often lack native integration, requiring adapters to
  align visual representations with pre-trained large language models (LLMs); (2)
  many are restricted to single-modal generation; (3) while some support
  multimodal generation, they rely on separate diffusion models for visual
  modeling and generation. To mitigate these limitations, we present Anole, an
  open, autoregressive, native large multimodal model for interleaved image-text
  generation. We build Anole from Meta AI's Chameleon, adopting an innovative
  fine-tuning strategy that is both data-efficient and parameter-efficient. Anole
  demonstrates high-quality, coherent multimodal generation capabilities. We have
  open-sourced our model, training framework, and instruction tuning data.
paper_ai_keywords: LMMs (Large Multimodal Models) adapters pre-trained large language models (LLMs) static single-modal generation multismodal generation diffusion models Anole autoregressive interleaved image-text generation Meta AI's Chameleon fine-tuning strategy data-efficient parameter-efficient multimodal generation high-quality coherent open-sourced model training framework instruction tuning data