title: OtterHD: A High-Resolution Multi-modality Model
summary: In this paper, we present OtterHD-8B, an innovative multimodal model evolved
  from Fuyu-8B, specifically engineered to interpret high-resolution visual
  inputs with granular precision. Unlike conventional models that are constrained
  by fixed-size vision encoders, OtterHD-8B boasts the ability to handle flexible
  input dimensions, ensuring its versatility across various inference
  requirements. Alongside this model, we introduce MagnifierBench, an evaluation
  framework designed to scrutinize models' ability to discern minute details and
  spatial relationships of small objects. Our comparative analysis reveals that
  while current leading models falter on this benchmark, OtterHD-8B, particularly
  when directly processing high-resolution inputs, outperforms its counterparts
  by a substantial margin. The findings illuminate the structural variances in
  visual information processing among different models and the influence that the
  vision encoders' pre-training resolution disparities have on model
  effectiveness within such benchmarks. Our study highlights the critical role of
  flexibility and high-resolution input capabilities in large multimodal models
  and also exemplifies the potential inherent in the Fuyu architecture's
  simplicity for handling complex visual data.
paper_authors._id: 654b0905980ac7a3d5985672
paper_authors.name: Bo Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-21T07:55:22.027Z
paper_authors._id: 62d3f7d84b0933c48f3cdd9c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62d3f7d84b0933c48f3cdd9c/rSoSyH0Td9fD9HVlyK7bh.jpeg
paper_authors.fullname: Bo Li
paper_authors.type: user
paper_authors.user: luodian
paper_authors._id: 654b0905980ac7a3d5985673
paper_authors.name: Peiyuan Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-08T08:27:22.064Z
paper_authors._id: 63565cc56d7fcf1bedb7d347
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63565cc56d7fcf1bedb7d347/XGcHP4VkO_oieA1gZ4IAX.jpeg
paper_authors.fullname: Zhang Peiyuan
paper_authors.type: user
paper_authors.user: PY007
paper_authors._id: 654b0905980ac7a3d5985674
paper_authors.name: Jingkang Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-08T08:42:36.122Z
paper_authors._id: 62b5777f593a2c49da69dc02
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1658152070753-62b5777f593a2c49da69dc02.jpeg
paper_authors.fullname: Jingkang Yang
paper_authors.type: user
paper_authors.user: Jingkang
paper_authors._id: 654b0905980ac7a3d5985675
paper_authors.name: Yuanhan Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-08T08:43:31.653Z
paper_authors._id: 62a993d80472c0b7f94027df
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62a993d80472c0b7f94027df/j5vp-IwLA2YBexylUHiQU.png
paper_authors.fullname: Zhang Yuanhan
paper_authors.type: user
paper_authors.user: ZhangYuanhan
paper_authors._id: 654b0905980ac7a3d5985676
paper_authors.name: Fanyi Pu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-08T08:43:45.026Z
paper_authors._id: 646e1ef5075bbcc48ddf21e8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/_vJC0zeVOIvaNV2R6toqg.jpeg
paper_authors.fullname: Pu Fanyi
paper_authors.type: user
paper_authors.user: pufanyi
paper_authors._id: 654b0905980ac7a3d5985677
paper_authors.name: Ziwei Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-08T08:43:54.424Z
paper_authors._id: 62ab1ac1d48b4d8b048a3473
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
paper_authors.fullname: Ziwei Liu
paper_authors.type: user
paper_authors.user: liuziwei7
paper_title: OtterHD: A High-Resolution Multi-modality Model
paper_summary: In this paper, we present OtterHD-8B, an innovative multimodal model evolved
  from Fuyu-8B, specifically engineered to interpret high-resolution visual
  inputs with granular precision. Unlike conventional models that are constrained
  by fixed-size vision encoders, OtterHD-8B boasts the ability to handle flexible
  input dimensions, ensuring its versatility across various inference
  requirements. Alongside this model, we introduce MagnifierBench, an evaluation
  framework designed to scrutinize models' ability to discern minute details and
  spatial relationships of small objects. Our comparative analysis reveals that
  while current leading models falter on this benchmark, OtterHD-8B, particularly
  when directly processing high-resolution inputs, outperforms its counterparts
  by a substantial margin. The findings illuminate the structural variances in
  visual information processing among different models and the influence that the
  vision encoders' pre-training resolution disparities have on model
  effectiveness within such benchmarks. Our study highlights the critical role of
  flexibility and high-resolution input capabilities in large multimodal models
  and also exemplifies the potential inherent in the Fuyu architecture's
  simplicity for handling complex visual data.
paper_ai_keywords: multimodal model Otter HD-8B Fuyu-8B high-resolution visual inputs vision encoders weight sharing inference requirements MagnifierBench evaluation framework minute details spatial relationships comparative analysis pre-training resolution model effectiveness flexibility large multimodal models Fuyu architecture