title: Jamba-1.5: Hybrid Transformer-Mamba Models at Scale
summary: We present Jamba-1.5, new instruction-tuned large language models based on
  our Jamba architecture. Jamba is a hybrid Transformer-Mamba mixture of experts
  architecture, providing high throughput and low memory usage across context
  lengths, while retaining the same or better quality as Transformer models. We
  release two model sizes: Jamba-1.5-Large, with 94B active parameters, and
  Jamba-1.5-Mini, with 12B active parameters. Both models are fine-tuned for a
  variety of conversational and instruction-following capabilties, and have an
  effective context length of 256K tokens, the largest amongst open-weight
  models. To support cost-effective inference, we introduce ExpertsInt8, a novel
  quantization technique that allows fitting Jamba-1.5-Large on a machine with 8
  80GB GPUs when processing 256K-token contexts without loss of quality. When
  evaluated on a battery of academic and chatbot benchmarks, Jamba-1.5 models
  achieve excellent results while providing high throughput and outperforming
  other open-weight models on long-context benchmarks. The model weights for both
  sizes are publicly available under the Jamba Open Model License and we release
  ExpertsInt8 as open source.
paper_authors._id: 66c801c3d5a538bbb9e6f962
paper_authors.name: Jamba Team
paper_authors._id: 66c801c3d5a538bbb9e6f963
paper_authors.name: Barak Lenz
paper_authors._id: 66c801c3d5a538bbb9e6f964
paper_authors.name: Alan Arazi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T14:09:52.602Z
paper_authors._id: 60d84af7eac5e05d4594f010
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60d84af7eac5e05d4594f010/KnGxUR7OUOAGg0S67tRaY.png
paper_authors.fullname: Alan Arazi
paper_authors.type: user
paper_authors.user: alana89
paper_authors._id: 66c801c3d5a538bbb9e6f965
paper_authors.name: Amir Bergman
paper_authors._id: 66c801c3d5a538bbb9e6f966
paper_authors.name: Avshalom Manevich
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-07T07:30:30.260Z
paper_authors._id: 6311d40c41ebdfa5f1196997
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6311d40c41ebdfa5f1196997/1TkHAAjtsO6oyBipyRi4n.png
paper_authors.fullname: Avshalom Manevich
paper_authors.type: user
paper_authors.user: avshalomman
paper_authors._id: 66c801c3d5a538bbb9e6f967
paper_authors.name: Barak Peleg
paper_authors._id: 66c801c3d5a538bbb9e6f968
paper_authors.name: Ben Aviram
paper_authors._id: 66c801c3d5a538bbb9e6f969
paper_authors.name: Chen Almagor
paper_authors._id: 66c801c3d5a538bbb9e6f96a
paper_authors.name: Clara Fridman
paper_authors._id: 66c801c3d5a538bbb9e6f96b
paper_authors.name: Dan Padnos
paper_authors._id: 66c801c3d5a538bbb9e6f96c
paper_authors.name: Daniel Gissin
paper_authors._id: 66c801c3d5a538bbb9e6f96d
paper_authors.name: Daniel Jannai
paper_authors._id: 66c801c3d5a538bbb9e6f96e
paper_authors.name: Dor Muhlgay
paper_authors._id: 66c801c3d5a538bbb9e6f96f
paper_authors.name: Dor Zimberg
paper_authors._id: 66c801c3d5a538bbb9e6f970
paper_authors.name: Edden M Gerber
paper_authors._id: 66c801c3d5a538bbb9e6f971
paper_authors.name: Elad Dolev
paper_authors._id: 66c801c3d5a538bbb9e6f972
paper_authors.name: Eran Krakovsky
paper_authors._id: 66c801c3d5a538bbb9e6f973
paper_authors.name: Erez Safahi
paper_authors._id: 66c801c3d5a538bbb9e6f974
paper_authors.name: Erez Schwartz
paper_authors._id: 66c801c3d5a538bbb9e6f975
paper_authors.name: Gal Cohen
paper_authors._id: 66c801c3d5a538bbb9e6f976
paper_authors.name: Gal Shachaf
paper_authors._id: 66c801c3d5a538bbb9e6f977
paper_authors.name: Haim Rozenblum
paper_authors._id: 66c801c3d5a538bbb9e6f978
paper_authors.name: Hofit Bata
paper_authors._id: 66c801c3d5a538bbb9e6f979
paper_authors.name: Ido Blass
paper_authors._id: 66c801c3d5a538bbb9e6f97a
paper_authors.name: Inbal Magar
paper_authors._id: 66c801c3d5a538bbb9e6f97b
paper_authors.name: Itay Dalmedigos
paper_authors._id: 66c801c3d5a538bbb9e6f97c
paper_authors.name: Jhonathan Osin
paper_authors._id: 66c801c3d5a538bbb9e6f97d
paper_authors.name: Julie Fadlon
paper_authors._id: 66c801c3d5a538bbb9e6f97e
paper_authors.name: Maria Rozman
paper_authors._id: 66c801c3d5a538bbb9e6f97f
paper_authors.name: Matan Danos
paper_authors._id: 66c801c3d5a538bbb9e6f980
paper_authors.name: Michael Gokhman
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-27T07:44:04.708Z
paper_authors._id: 60ede03f36e9ceda399a423e
paper_authors.avatarUrl: /avatars/da4794bfca8c430d271fcb50a19da28f.svg
paper_authors.fullname: Michael Gokhman
paper_authors.type: user
paper_authors.user: michael-go
paper_authors._id: 66c801c3d5a538bbb9e6f981
paper_authors.name: Mor Zusman
paper_authors._id: 66c801c3d5a538bbb9e6f982
paper_authors.name: Naama Gidron
paper_authors._id: 66c801c3d5a538bbb9e6f983
paper_authors.name: Nir Ratner
paper_authors._id: 66c801c3d5a538bbb9e6f984
paper_authors.name: Noam Gat
paper_authors._id: 66c801c3d5a538bbb9e6f985
paper_authors.name: Noam Rozen
paper_authors._id: 66c801c3d5a538bbb9e6f986
paper_authors.name: Oded Fried
paper_authors._id: 66c801c3d5a538bbb9e6f987
paper_authors.name: Ohad Leshno
paper_authors._id: 66c801c3d5a538bbb9e6f988
paper_authors.name: Omer Antverg
paper_authors._id: 66c801c3d5a538bbb9e6f989
paper_authors.name: Omri Abend
paper_authors._id: 66c801c3d5a538bbb9e6f98a
paper_authors.name: Opher Lieber
paper_authors._id: 66c801c3d5a538bbb9e6f98b
paper_authors.name: Or Dagan
paper_authors._id: 66c801c3d5a538bbb9e6f98c
paper_authors.name: Orit Cohavi
paper_authors._id: 66c801c3d5a538bbb9e6f98d
paper_authors.name: Raz Alon
paper_authors._id: 66c801c3d5a538bbb9e6f98e
paper_authors.name: Ro'i Belson
paper_authors._id: 66c801c3d5a538bbb9e6f98f
paper_authors.name: Roi Cohen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-02T08:04:08.536Z
paper_authors._id: 64b7b79a8ba7d6c922d75144
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64b7b79a8ba7d6c922d75144/S6WUOCj1_bKc4Gv62uSaa.jpeg
paper_authors.fullname: Roi Cohen
paper_authors.type: user
paper_authors.user: roicohennn
paper_authors._id: 66c801c3d5a538bbb9e6f990
paper_authors.name: Rom Gilad
paper_authors._id: 66c801c3d5a538bbb9e6f991
paper_authors.name: Roman Glozman
paper_authors._id: 66c801c3d5a538bbb9e6f992
paper_authors.name: Shahar Lev
paper_authors._id: 66c801c3d5a538bbb9e6f993
paper_authors.name: Shaked Meirom
paper_authors._id: 66c801c3d5a538bbb9e6f994
paper_authors.name: Tal Delbari
paper_authors._id: 66c801c3d5a538bbb9e6f995
paper_authors.name: Tal Ness
paper_authors._id: 66c801c3d5a538bbb9e6f996
paper_authors.name: Tomer Asida
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-27T07:43:57.165Z
paper_authors._id: 65f2d8dfab7d7db7f5d78900
paper_authors.avatarUrl: /avatars/feaa4973b6a6741c159bb2bc94d1ba59.svg
paper_authors.fullname: Tomer Asida
paper_authors.type: user
paper_authors.user: tomeras1
paper_authors._id: 66c801c3d5a538bbb9e6f997
paper_authors.name: Tom Ben Gal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-29T15:22:12.956Z
paper_authors._id: 65e6e40b5abdebc53059f2bf
paper_authors.avatarUrl: /avatars/cd71b74d4d237f869cdbbff3361e88dd.svg
paper_authors.fullname: Tom Ben Gal
paper_authors.type: user
paper_authors.user: tombg
paper_authors._id: 66c801c3d5a538bbb9e6f998
paper_authors.name: Tom Braude
paper_authors._id: 66c801c3d5a538bbb9e6f999
paper_authors.name: Uriya Pumerantz
paper_authors._id: 66c801c3d5a538bbb9e6f99a
paper_authors.name: Yehoshua Cohen
paper_authors._id: 66c801c3d5a538bbb9e6f99b
paper_authors.name: Yonatan Belinkov
paper_authors._id: 66c801c3d5a538bbb9e6f99c
paper_authors.name: Yuval Globerson
paper_authors._id: 66c801c3d5a538bbb9e6f99d
paper_authors.name: Yuval Peleg Levy
paper_authors._id: 66c801c3d5a538bbb9e6f99e
paper_authors.name: Yoav Shoham
paper_title: Jamba-1.5: Hybrid Transformer-Mamba Models at Scale
paper_summary: We present Jamba-1.5, new instruction-tuned large language models based on
  our Jamba architecture. Jamba is a hybrid Transformer-Mamba mixture of experts
  architecture, providing high throughput and low memory usage across context
  lengths, while retaining the same or better quality as Transformer models. We
  release two model sizes: Jamba-1.5-Large, with 94B active parameters, and
  Jamba-1.5-Mini, with 12B active parameters. Both models are fine-tuned for a
  variety of conversational and instruction-following capabilties, and have an
  effective context length of 256K tokens, the largest amongst open-weight
  models. To support cost-effective inference, we introduce ExpertsInt8, a novel
  quantization technique that allows fitting Jamba-1.5-Large on a machine with 8
  80GB GPUs when processing 256K-token contexts without loss of quality. When
  evaluated on a battery of academic and chatbot benchmarks, Jamba-1.5 models
  achieve excellent results while providing high throughput and outperforming
  other open-weight models on long-context benchmarks. The model weights for both
  sizes are publicly available under the Jamba Open Model License and we release
  ExpertsInt8 as open source.
paper_ai_summary: Jamba-1.5, a hybrid Transformer-Mamba model, offers high throughput and low memory usage while achieving high quality in instruction-following tasks, supported by ExpertsInt8 quantization.
paper_ai_keywords: Transformer Mamba mixture of experts Jamba architecture ExpertsInt8 quantization technique