Ming-Omni: A Unified Multimodal Model for Perception and Generation
We propose Ming-Omni, a unified multimodal model capable of processing
images, text, audio, and video, while demonstrating strong proficiency in both
speech and image generation. Ming-Omni employs dedicated encoders to extract
tokens from different modalities, which are then processed by Ling, an MoE
architecture equipped with newly proposed modality-specific routers. This
design enables a single model to efficiently process and fuse multimodal inputs
within a unified framework, thereby facilitating diverse tasks without
requiring separate models, task-specific fine-tuning, or structural redesign.
Importantly, Ming-Omni extends beyond conventional multimodal models by
supporting audio and image generation. This is achieved through the integration
of an advanced audio decoder for natural-sounding speech and Ming-Lite-Uni for
high-quality image generation, which also allow the model to engage in
context-aware chatting, perform text-to-speech conversion, and conduct
versatile image editing. Our experimental results showcase Ming-Omni offers a
powerful solution for unified perception and generation across all modalities.
Notably, our proposed Ming-Omni is the first open-source model we are aware of
to match GPT-4o in modality support, and we release all code and model weights
to encourage further research and development in the community.
684ae277dbd21a9cc27b118d
Inclusion AI
684ae277dbd21a9cc27b118e
Biao Gong
admin_assigned
2025-06-14T06:24:13.239Z
644fcbea4f7316588267dc80
https://cdn-avatars.huggingface.co/v1/production/uploads/644fcbea4f7316588267dc80/w8-2Gkaw9BN9VzppNXrTP.jpeg
Biao Gong
user
BiaoGong
684ae277dbd21a9cc27b118f
Cheng Zou
684ae277dbd21a9cc27b1190
Chuanyang Zheng
admin_assigned
2025-06-14T06:42:02.773Z
65019cc870367843160fbb33
/avatars/f5455482fe9efbdeeea1bd3a3c119f02.svg
ZhengChuanyang
user
BillionZheng
684ae277dbd21a9cc27b1191
Chunluan Zhou
684ae277dbd21a9cc27b1192
Canxiang Yan
684ae277dbd21a9cc27b1193
Chunxiang Jin
684ae277dbd21a9cc27b1194
Chunjie Shen
684ae277dbd21a9cc27b1195
Dandan Zheng
admin_assigned
2025-06-14T06:42:39.174Z
65dd699a89a2a760d15f7d35
/avatars/e098b56c413d147d1f38cf33a4b0ecde.svg
Dandan Zheng
user
zhengdd0422
684ae277dbd21a9cc27b1196
Fudong Wang
admin_assigned
2025-06-14T06:42:46.033Z
658b8ec75208d1ca0e2e72ca
/avatars/aab882fe4947097ca67a692394475f59.svg
Fudong Wang
user
fudongwang
684ae277dbd21a9cc27b1197
Furong Xu
admin_assigned
2025-06-14T06:42:56.425Z
6806024861c16bb93660510a
/avatars/ff0b288fa90f35f67b3b6f7b1e184474.svg
xu furong
user
booyoungxu
684ae277dbd21a9cc27b1198
GuangMing Yao
684ae277dbd21a9cc27b1199
Jun Zhou
684ae277dbd21a9cc27b119a
Jingdong Chen
684ae277dbd21a9cc27b119b
Jianxin Sun
684ae277dbd21a9cc27b119c
Jiajia Liu
684ae277dbd21a9cc27b119d
Jianjiang Zhu
684ae277dbd21a9cc27b119e
Jun Peng
684ae277dbd21a9cc27b119f
Kaixiang Ji
684ae277dbd21a9cc27b11a0
Kaiyou Song
684ae277dbd21a9cc27b11a1
Kaimeng Ren
684ae277dbd21a9cc27b11a2
Libin Wang
684ae277dbd21a9cc27b11a3
Lixiang Ru
684ae277dbd21a9cc27b11a4
Lele Xie
684ae277dbd21a9cc27b11a5
Longhua Tan
684ae277dbd21a9cc27b11a6
Lyuxin Xue
684ae277dbd21a9cc27b11a7
Lan Wang
684ae277dbd21a9cc27b11a8
Mochen Bai
684ae277dbd21a9cc27b11a9
Ning Gao
684ae277dbd21a9cc27b11aa
Pei Chen
684ae277dbd21a9cc27b11ab
Qingpei Guo
684ae277dbd21a9cc27b11ac
Qinglong Zhang
684ae277dbd21a9cc27b11ad
Qiang Xu
684ae277dbd21a9cc27b11ae
Rui Liu
684ae277dbd21a9cc27b11af
Ruijie Xiong
684ae277dbd21a9cc27b11b0
Sirui Gao
684ae277dbd21a9cc27b11b1
Tinghao Liu
684ae277dbd21a9cc27b11b2
Taisong Li
684ae277dbd21a9cc27b11b3
Weilong Chai
684ae277dbd21a9cc27b11b4
Xinyu Xiao
684ae277dbd21a9cc27b11b5
Xiaomei Wang
684ae277dbd21a9cc27b11b6
Xiaoxue Chen
684ae277dbd21a9cc27b11b7
Xiao Lu
684ae277dbd21a9cc27b11b8
Xiaoyu Li
684ae277dbd21a9cc27b11b9
Xingning Dong
684ae277dbd21a9cc27b11ba
Xuzheng Yu
684ae277dbd21a9cc27b11bb
Yi Yuan
684ae277dbd21a9cc27b11bc
Yuting Gao
684ae277dbd21a9cc27b11bd
Yunxiao Sun
684ae277dbd21a9cc27b11be
Yipeng Chen
684ae277dbd21a9cc27b11bf
Yifei Wu
684ae277dbd21a9cc27b11c0
Yongjie Lyu
684ae277dbd21a9cc27b11c1
Ziping Ma
claimed_verified
2025-06-16T07:17:01.575Z
66b0780a87f605ac4dbaca64
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/-B1ovhWJW2dO3n8ug1dY_.png
ZipingMa
user
ZipingMa
684ae277dbd21a9cc27b11c2
Zipeng Feng
684ae277dbd21a9cc27b11c3
Zhijiang Fang
684ae277dbd21a9cc27b11c4
Zhihao Qiu
684ae277dbd21a9cc27b11c5
Ziyuan Huang
684ae277dbd21a9cc27b11c6
Zhengyu He
admin_assigned
2025-06-14T06:46:20.958Z
6507c37a38b7f6bcfaaa3ad6
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/T8HXjz9fJcToQ9Cp40nWo.jpeg
Zhengyu He
user
CognacDrop
Ming-Omni: A Unified Multimodal Model for Perception and Generation
We propose Ming-Omni, a unified multimodal model capable of processing
images, text, audio, and video, while demonstrating strong proficiency in both
speech and image generation. Ming-Omni employs dedicated encoders to extract
tokens from different modalities, which are then processed by Ling, an MoE
architecture equipped with newly proposed modality-specific routers. This
design enables a single model to efficiently process and fuse multimodal inputs
within a unified framework, thereby facilitating diverse tasks without
requiring separate models, task-specific fine-tuning, or structural redesign.
Importantly, Ming-Omni extends beyond conventional multimodal models by
supporting audio and image generation. This is achieved through the integration
of an advanced audio decoder for natural-sounding speech and Ming-Lite-Uni for
high-quality image generation, which also allow the model to engage in
context-aware chatting, perform text-to-speech conversion, and conduct
versatile image editing. Our experimental results showcase Ming-Omni offers a
powerful solution for unified perception and generation across all modalities.
Notably, our proposed Ming-Omni is the first open-source model we are aware of
to match GPT-4o in modality support, and we release all code and model weights
to encourage further research and development in the community.
Ming-Omni is a unified multimodal model with dedicated encoders and modality-specific routers that can process images, text, audio, and video, and performs tasks like speech and image generation, context-aware chatting, and versatile image editing.
multimodal model encoders tokens MoE architecture modality-specific routers audio decoder Ming-Lite-Uni context-aware chatting text-to-speech conversion image editing unified perception generation open-source