title: LLaVaOLMoBitnet1B: Ternary LLM goes Multimodal!
summary: Multimodal Large Language Models (MM-LLMs) have seen significant advancements
  in the last year, demonstrating impressive performance across tasks. However,
  to truly democratize AI, models must exhibit strong capabilities and be able to
  run efficiently on small compute footprints accessible by most. Part of this
  quest, we introduce LLaVaOLMoBitnet1B - the first Ternary Multimodal LLM
  capable of accepting Image(s)+Text inputs to produce coherent textual
  responses. The model is fully open-sourced along with training scripts to
  encourage further research in this space. This accompanying technical report
  highlights the training process, evaluation details, challenges associated with
  ternary models and future opportunities. Link to the model:
  https://huggingface.co/IntelLabs/LlavaOLMoBitnet1B
paper_authors._id: 66cd46147ab136deb3862f5e
paper_authors.name: Jainaveen Sundaram
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-27T20:42:26.586Z
paper_authors._id: 615f72d3205e10d5c3903df3
paper_authors.avatarUrl: /avatars/c85352ad1b858fd6c1629ddea06d0dea.svg
paper_authors.fullname: Naveen Sp
paper_authors.type: user
paper_authors.user: naveensp
paper_authors._id: 66cd46147ab136deb3862f5f
paper_authors.name: Ravishankar Iyer
paper_title: LLaVaOLMoBitnet1B: Ternary LLM goes Multimodal!
paper_summary: Multimodal Large Language Models (MM-LLMs) have seen significant advancements
  in the last year, demonstrating impressive performance across tasks. However,
  to truly democratize AI, models must exhibit strong capabilities and be able to
  run efficiently on small compute footprints accessible by most. Part of this
  quest, we introduce LLaVaOLMoBitnet1B - the first Ternary Multimodal LLM
  capable of accepting Image(s)+Text inputs to produce coherent textual
  responses. The model is fully open-sourced along with training scripts to
  encourage further research in this space. This accompanying technical report
  highlights the training process, evaluation details, challenges associated with
  ternary models and future opportunities. Link to the model:
  https://huggingface.co/IntelLabs/LlavaOLMoBitnet1B
paper_ai_keywords: Multimodal Large Language Models (MM-LLMs) Ternary Multimodal LLM Image(s)+Text inputs coherent textual responses training scripts