title: POINTS1.5: Building a Vision-Language Model towards Real World
    Applications
summary: Vision-language models have made significant strides recently, demonstrating
  superior performance across a range of tasks, e.g. optical character
  recognition and complex diagram analysis. Building on this trend, we introduce
  a new vision-language model, POINTS1.5, designed to excel in various real-world
  applications. POINTS1.5 is an enhancement of POINTS1.0 and incorporates several
  key innovations: i) We replace the original CLIP vision encoder, which had a
  fixed image resolution, with a NaViT-style vision encoder that supports native
  dynamic high resolution. This allows POINTS1.5 to process images of any
  resolution without needing to split them into tiles. ii) We add bilingual
  support to POINTS1.5, significantly enhancing its capability in Chinese. Due to
  the scarcity of open-source Chinese datasets for vision-language models, we
  collect numerous images from the Internet and annotate them using a combination
  of manual and automatic methods. iii) We propose a set of rigorous filtering
  methods for visual instruction tuning datasets. We comprehensively evaluate all
  these filtering methods, and choose the most effective ones to obtain the final
  visual instruction tuning set. Thanks to these innovations, POINTS1.5
  significantly outperforms POINTS1.0 and demonstrates strong performance across
  a range of real-world applications. Notably, POINTS1.5-7B is trained on fewer
  than 4 billion tokens and ranks first on the OpenCompass leaderboard among
  models with fewer than 10 billion parameters
paper_authors._id: 675a62917f9a1b78132efc33
paper_authors.name: Yuan Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-12T08:28:51.562Z
paper_authors._id: 64d47a7a508a6313e33faedd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64d47a7a508a6313e33faedd/8DrbXdxA-sKZb4jgNhvY1.jpeg
paper_authors.fullname: Yuan Liu
paper_authors.type: user
paper_authors.user: YuanLiuuuuuu
paper_authors._id: 675a62917f9a1b78132efc34
paper_authors.name: Le Tian
paper_authors._id: 675a62917f9a1b78132efc35
paper_authors.name: Xiao Zhou
paper_authors._id: 675a62917f9a1b78132efc36
paper_authors.name: Xinyu Gao
paper_authors._id: 675a62917f9a1b78132efc37
paper_authors.name: Kavio Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-12T08:29:26.282Z
paper_authors._id: 663f288e9cb0add00b96bf45
paper_authors.avatarUrl: /avatars/cd90b9490c439703cb3b52ccbc491199.svg
paper_authors.fullname: kavioyu
paper_authors.type: user
paper_authors.user: kavio
paper_authors._id: 675a62917f9a1b78132efc38
paper_authors.name: Yang Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-12T08:29:33.015Z
paper_authors._id: 646aefc235c7a57f936cfa31
paper_authors.avatarUrl: /avatars/fc55fadf7260f8d9e164ddafb5e2b8dc.svg
paper_authors.fullname: Yang Yu
paper_authors.type: user
paper_authors.user: yangyu1
paper_authors._id: 675a62917f9a1b78132efc39
paper_authors.name: Jie Zhou
paper_title: POINTS1.5: Building a Vision-Language Model towards Real World
    Applications
paper_summary: Vision-language models have made significant strides recently, demonstrating
  superior performance across a range of tasks, e.g. optical character
  recognition and complex diagram analysis. Building on this trend, we introduce
  a new vision-language model, POINTS1.5, designed to excel in various real-world
  applications. POINTS1.5 is an enhancement of POINTS1.0 and incorporates several
  key innovations: i) We replace the original CLIP vision encoder, which had a
  fixed image resolution, with a NaViT-style vision encoder that supports native
  dynamic high resolution. This allows POINTS1.5 to process images of any
  resolution without needing to split them into tiles. ii) We add bilingual
  support to POINTS1.5, significantly enhancing its capability in Chinese. Due to
  the scarcity of open-source Chinese datasets for vision-language models, we
  collect numerous images from the Internet and annotate them using a combination
  of manual and automatic methods. iii) We propose a set of rigorous filtering
  methods for visual instruction tuning datasets. We comprehensively evaluate all
  these filtering methods, and choose the most effective ones to obtain the final
  visual instruction tuning set. Thanks to these innovations, POINTS1.5
  significantly outperforms POINTS1.0 and demonstrates strong performance across
  a range of real-world applications. Notably, POINTS1.5-7B is trained on fewer
  than 4 billion tokens and ranks first on the OpenCompass leaderboard among
  models with fewer than 10 billion parameters
paper_ai_keywords: CLIP vision encoder NaViT-style vision encoder dynamic high resolution bilingual support visual instruction tuning filtering methods