title: General Object Foundation Model for Images and Videos at Scale
summary: We present GLEE in this work, an object-level foundation model for locating
  and identifying objects in images and videos. Through a unified framework, GLEE
  accomplishes detection, segmentation, tracking, grounding, and identification
  of arbitrary objects in the open world scenario for various object perception
  tasks. Adopting a cohesive learning strategy, GLEE acquires knowledge from
  diverse data sources with varying supervision levels to formulate general
  object representations, excelling in zero-shot transfer to new data and tasks.
  Specifically, we employ an image encoder, text encoder, and visual prompter to
  handle multi-modal inputs, enabling to simultaneously solve various
  object-centric downstream tasks while maintaining state-of-the-art performance.
  Demonstrated through extensive training on over five million images from
  diverse benchmarks, GLEE exhibits remarkable versatility and improved
  generalization performance, efficiently tackling downstream tasks without the
  need for task-specific adaptation. By integrating large volumes of
  automatically labeled data, we further enhance its zero-shot generalization
  capabilities. Additionally, GLEE is capable of being integrated into Large
  Language Models, serving as a foundational model to provide universal
  object-level information for multi-modal tasks. We hope that the versatility
  and universality of our method will mark a significant step in the development
  of efficient visual foundation models for AGI systems. The model and code will
  be released at https://glee-vision.github.io .
paper_authors._id: 657baf29f81f6b44b89fe494
paper_authors.name: Junfeng Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-15T13:28:39.118Z
paper_authors._id: 6569de56046899997ba5ac4e
paper_authors.avatarUrl: /avatars/287ee4c36a9edf2540eed779fd7d41e2.svg
paper_authors.fullname: Junfeng Wu
paper_authors.type: user
paper_authors.user: Junfeng5
paper_authors._id: 657baf29f81f6b44b89fe495
paper_authors.name: Yi Jiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-05T09:33:09.388Z
paper_authors._id: 6344dcb1cd37e44d9ed46508
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6344dcb1cd37e44d9ed46508/J92UKSxKR3iziD2WJfih4.jpeg
paper_authors.fullname: Yi Jiang
paper_authors.type: user
paper_authors.user: JiangYi
paper_authors._id: 657baf29f81f6b44b89fe496
paper_authors.name: Qihao Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-15T13:15:59.280Z
paper_authors._id: 639f1e519f1f2baab2f00d22
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/639f1e519f1f2baab2f00d22/pFjd51WZuVZ3A11rItvmk.jpeg
paper_authors.fullname: Qihao Liu
paper_authors.type: user
paper_authors.user: QHL067
paper_authors._id: 657baf29f81f6b44b89fe497
paper_authors.name: Zehuan Yuan
paper_authors._id: 657baf29f81f6b44b89fe498
paper_authors.name: Xiang Bai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-15T13:16:24.948Z
paper_authors._id: 641790e2f1e86908935d82a0
paper_authors.avatarUrl: /avatars/ced7a137c6344c74b7ac0d5c84833fc8.svg
paper_authors.fullname: Xiang Bai
paper_authors.type: user
paper_authors.user: baixianger
paper_authors._id: 657baf29f81f6b44b89fe499
paper_authors.name: Song Bai
paper_title: General Object Foundation Model for Images and Videos at Scale
paper_summary: We present GLEE in this work, an object-level foundation model for locating
  and identifying objects in images and videos. Through a unified framework, GLEE
  accomplishes detection, segmentation, tracking, grounding, and identification
  of arbitrary objects in the open world scenario for various object perception
  tasks. Adopting a cohesive learning strategy, GLEE acquires knowledge from
  diverse data sources with varying supervision levels to formulate general
  object representations, excelling in zero-shot transfer to new data and tasks.
  Specifically, we employ an image encoder, text encoder, and visual prompter to
  handle multi-modal inputs, enabling to simultaneously solve various
  object-centric downstream tasks while maintaining state-of-the-art performance.
  Demonstrated through extensive training on over five million images from
  diverse benchmarks, GLEE exhibits remarkable versatility and improved
  generalization performance, efficiently tackling downstream tasks without the
  need for task-specific adaptation. By integrating large volumes of
  automatically labeled data, we further enhance its zero-shot generalization
  capabilities. Additionally, GLEE is capable of being integrated into Large
  Language Models, serving as a foundational model to provide universal
  object-level information for multi-modal tasks. We hope that the versatility
  and universality of our method will mark a significant step in the development
  of efficient visual foundation models for AGI systems. The model and code will
  be released at https://glee-vision.github.io .
paper_ai_summary: GLEE is an object-level foundation model for tasks like detection, segmentation, tracking, and identification, demonstrating zero-shot transfer and state-of-the-art performance.
paper_ai_keywords: object-level foundation model detection segmentation tracking grounding identification open world scenario object perception tasks cohesive learning strategy image encoder text encoder visual prompter multi-modal inputs downstream tasks zero-shot transfer state-of-the-art performance general object representations generalization performance large volumes of automatically labeled data Large Language Models AGI systems