title: OpenVision: A Fully-Open, Cost-Effective Family of Advanced Vision
    Encoders for Multimodal Learning
summary: OpenAI's CLIP, released in early 2021, have long been the go-to choice of
  vision encoder for building multimodal foundation models. Although recent
  alternatives such as SigLIP have begun to challenge this status quo, to our
  knowledge none are fully open: their training data remains proprietary and/or
  their training recipes are not released. This paper fills this gap with
  OpenVision, a fully-open, cost-effective family of vision encoders that match
  or surpass the performance of OpenAI's CLIP when integrated into multimodal
  frameworks like LLaVA. OpenVision builds on existing works -- e.g., CLIPS for
  training framework and Recap-DataComp-1B for training data -- while revealing
  multiple key insights in enhancing encoder quality and showcasing practical
  benefits in advancing multimodal models. By releasing vision encoders spanning
  from 5.9M to 632.1M parameters, OpenVision offers practitioners a flexible
  trade-off between capacity and efficiency in building multimodal models: larger
  models deliver enhanced multimodal performance, while smaller versions enable
  lightweight, edge-ready multimodal deployments.
paper_authors._id: 681d1be954749869547df71e
paper_authors.name: Xianhang Li
paper_authors._id: 681d1be954749869547df71f
paper_authors.name: Yanqing Liu
paper_authors._id: 681d1be954749869547df720
paper_authors.name: Haoqin Tu
paper_authors._id: 681d1be954749869547df721
paper_authors.name: Hongru Zhu
paper_authors._id: 681d1be954749869547df722
paper_authors.name: Cihang Xie
paper_title: OpenVision: A Fully-Open, Cost-Effective Family of Advanced Vision
    Encoders for Multimodal Learning
paper_summary: OpenAI's CLIP, released in early 2021, have long been the go-to choice of
  vision encoder for building multimodal foundation models. Although recent
  alternatives such as SigLIP have begun to challenge this status quo, to our
  knowledge none are fully open: their training data remains proprietary and/or
  their training recipes are not released. This paper fills this gap with
  OpenVision, a fully-open, cost-effective family of vision encoders that match
  or surpass the performance of OpenAI's CLIP when integrated into multimodal
  frameworks like LLaVA. OpenVision builds on existing works -- e.g., CLIPS for
  training framework and Recap-DataComp-1B for training data -- while revealing
  multiple key insights in enhancing encoder quality and showcasing practical
  benefits in advancing multimodal models. By releasing vision encoders spanning
  from 5.9M to 632.1M parameters, OpenVision offers practitioners a flexible
  trade-off between capacity and efficiency in building multimodal models: larger
  models deliver enhanced multimodal performance, while smaller versions enable
  lightweight, edge-ready multimodal deployments.
paper_ai_summary: OpenVision provides a fully-open, cost-effective family of vision encoders that matches or surpasses OpenAI's CLIP in multimodal frameworks, offering flexible trade-offs between model size and performance.
paper_ai_keywords: CLIP SigLIP multimodal foundation models LLaVA Recap-DataComp-1B vision encoders