title: CogVLM2: Visual Language Models for Image and Video Understanding
summary: Beginning with VisualGLM and CogVLM, we are continuously exploring VLMs in
  pursuit of enhanced vision-language fusion, efficient higher-resolution
  architecture, and broader modalities and applications. Here we propose the
  CogVLM2 family, a new generation of visual language models for image and video
  understanding including CogVLM2, CogVLM2-Video and GLM-4V. As an image
  understanding model, CogVLM2 inherits the visual expert architecture with
  improved training recipes in both pre-training and post-training stages,
  supporting input resolution up to 1344 times 1344 pixels. As a video
  understanding model, CogVLM2-Video integrates multi-frame input with timestamps
  and proposes automated temporal grounding data construction. Notably, CogVLM2
  family has achieved state-of-the-art results on benchmarks like MMBench,
  MM-Vet, TextVQA, MVBench and VCGBench. All models are open-sourced in
  https://github.com/THUDM/CogVLM2 and https://github.com/THUDM/GLM-4,
  contributing to the advancement of the field.
paper_authors._id: 66d1255a3b8f08e1b2de17be
paper_authors.name: Wenyi Hong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T07:19:24.884Z
paper_authors._id: 62ecd24cb8764c7738ef2793
paper_authors.avatarUrl: /avatars/c1b80b5c55f9d652c1aaac7919e1fa32.svg
paper_authors.fullname: Wenyi Hong
paper_authors.type: user
paper_authors.user: wenyi
paper_authors._id: 66d1255a3b8f08e1b2de17bf
paper_authors.name: Weihan Wang
paper_authors._id: 66d1255a3b8f08e1b2de17c0
paper_authors.name: Ming Ding
paper_authors._id: 66d1255a3b8f08e1b2de17c1
paper_authors.name: Wenmeng Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T07:24:30.771Z
paper_authors._id: 649d3d271bafbcc83acec930
paper_authors.avatarUrl: /avatars/0c42aabf4c6601686c22cc1308c318de.svg
paper_authors.fullname: Wenmeng Yu
paper_authors.type: user
paper_authors.user: iyuge2
paper_authors._id: 66d1255a3b8f08e1b2de17c2
paper_authors.name: Qingsong Lv
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T07:24:37.541Z
paper_authors._id: 61bd8ac694bd578f7286181b
paper_authors.avatarUrl: /avatars/ff14b64339217858927ae01c2b2fa28d.svg
paper_authors.fullname: Qingsong Lv
paper_authors.type: user
paper_authors.user: qingsonglv
paper_authors._id: 66d1255a3b8f08e1b2de17c3
paper_authors.name: Yan Wang
paper_authors._id: 66d1255a3b8f08e1b2de17c4
paper_authors.name: Yean Cheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T07:27:04.116Z
paper_authors._id: 65acc5afe2a2c8635614de43
paper_authors.avatarUrl: /avatars/c5fce792792cc0b52ed7475d72460c58.svg
paper_authors.fullname: Yean Cheng
paper_authors.type: user
paper_authors.user: LiquidAmmonia
paper_authors._id: 66d1255a3b8f08e1b2de17c5
paper_authors.name: Shiyu Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T07:27:10.517Z
paper_authors._id: 6406db5cd684369027166986
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6406db5cd684369027166986/Zl-orrGcbY0RbfjfKszn1.jpeg
paper_authors.fullname: Shiyu Huang
paper_authors.type: user
paper_authors.user: ShiyuHuang
paper_authors._id: 66d1255a3b8f08e1b2de17c6
paper_authors.name: Junhui Ji
paper_authors._id: 66d1255a3b8f08e1b2de17c7
paper_authors.name: Zhao Xue
paper_authors._id: 66d1255a3b8f08e1b2de17c8
paper_authors.name: Lei Zhao
paper_authors._id: 66d1255a3b8f08e1b2de17c9
paper_authors.name: Zhuoyi Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T07:28:04.753Z
paper_authors._id: 6466d1640ed2f7a8cba87503
paper_authors.avatarUrl: /avatars/652746e63dfeb5154ae7d34039d1a485.svg
paper_authors.fullname: Zhuoyi Yang
paper_authors.type: user
paper_authors.user: zyyangzy
paper_authors._id: 66d1255a3b8f08e1b2de17ca
paper_authors.name: Xiaotao Gu
paper_authors._id: 66d1255a3b8f08e1b2de17cb
paper_authors.name: Xiaohan Zhang
paper_authors._id: 66d1255a3b8f08e1b2de17cc
paper_authors.name: Guanyu Feng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T07:54:14.645Z
paper_authors._id: 64d996d6bcab729cb400cb70
paper_authors.avatarUrl: /avatars/3db57c8b643fba8302ed39d8cf0f4ddb.svg
paper_authors.fullname: Guanyu Feng
paper_authors.type: user
paper_authors.user: jiguanglizipao
paper_authors._id: 66d1255a3b8f08e1b2de17cd
paper_authors.name: Da Yin
paper_authors._id: 66d1255a3b8f08e1b2de17ce
paper_authors.name: Zihan Wang
paper_authors._id: 66d1255a3b8f08e1b2de17cf
paper_authors.name: Ji Qi
paper_authors._id: 66d1255a3b8f08e1b2de17d0
paper_authors.name: Xixuan Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T07:50:05.317Z
paper_authors._id: 63f36b504745321de3510823
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1679311624745-63f36b504745321de3510823.jpeg
paper_authors.fullname: Song XiXuan
paper_authors.type: user
paper_authors.user: DrSong
paper_authors._id: 66d1255a3b8f08e1b2de17d1
paper_authors.name: Peng Zhang
paper_authors._id: 66d1255a3b8f08e1b2de17d2
paper_authors.name: Debing Liu
paper_authors._id: 66d1255a3b8f08e1b2de17d3
paper_authors.name: Bin Xu
paper_authors._id: 66d1255a3b8f08e1b2de17d4
paper_authors.name: Juanzi Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T07:49:11.275Z
paper_authors._id: 65df8cbc2705d9672f55d1aa
paper_authors.avatarUrl: /avatars/63e46f15bb76bd9d4508fd0f54f39829.svg
paper_authors.fullname: Juanzi Li
paper_authors.type: user
paper_authors.user: juanli
paper_authors._id: 66d1255a3b8f08e1b2de17d5
paper_authors.name: Yuxiao Dong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T07:49:02.883Z
paper_authors._id: 640e73bdfdeaae1390857b62
paper_authors.avatarUrl: /avatars/cd6779e30f716002a7838ed93d5c0754.svg
paper_authors.fullname: Yuxiao Dong
paper_authors.type: user
paper_authors.user: yuxiaod
paper_authors._id: 66d1255a3b8f08e1b2de17d6
paper_authors.name: Jie Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T07:48:54.483Z
paper_authors._id: 640dff05474aa6f89556677e
paper_authors.avatarUrl: /avatars/1b4591c7322d649c797b3125148f1915.svg
paper_authors.fullname: Jie Tang
paper_authors.type: user
paper_authors.user: jerytang
paper_title: CogVLM2: Visual Language Models for Image and Video Understanding
paper_summary: Beginning with VisualGLM and CogVLM, we are continuously exploring VLMs in
  pursuit of enhanced vision-language fusion, efficient higher-resolution
  architecture, and broader modalities and applications. Here we propose the
  CogVLM2 family, a new generation of visual language models for image and video
  understanding including CogVLM2, CogVLM2-Video and GLM-4V. As an image
  understanding model, CogVLM2 inherits the visual expert architecture with
  improved training recipes in both pre-training and post-training stages,
  supporting input resolution up to 1344 times 1344 pixels. As a video
  understanding model, CogVLM2-Video integrates multi-frame input with timestamps
  and proposes automated temporal grounding data construction. Notably, CogVLM2
  family has achieved state-of-the-art results on benchmarks like MMBench,
  MM-Vet, TextVQA, MVBench and VCGBench. All models are open-sourced in
  https://github.com/THUDM/CogVLM2 and https://github.com/THUDM/GLM-4,
  contributing to the advancement of the field.
paper_ai_summary: The CogVLM2 family of visual language models, including CogVLM2, CogVLM2-Video, and GLM-4V, advances image and video understanding with enhanced architectures and training methods.
paper_ai_keywords: visual language models CogVLM2 CogVLM2-Video GLM-4V visual expert architecture pre-training post-training input resolution multi-frame input timestamps automated temporal grounding MMBench MM-Vet TextVQA MVBench VCGBench