title: Florence-2: Advancing a Unified Representation for a Variety of Vision
    Tasks
summary: We introduce Florence-2, a novel vision foundation model with a unified,
  prompt-based representation for a variety of computer vision and
  vision-language tasks. While existing large vision models excel in transfer
  learning, they struggle to perform a diversity of tasks with simple
  instructions, a capability that implies handling the complexity of various
  spatial hierarchy and semantic granularity. Florence-2 was designed to take
  text-prompt as task instructions and generate desirable results in text forms,
  whether it be captioning, object detection, grounding or segmentation. This
  multi-task learning setup demands large-scale, high-quality annotated data. To
  this end, we co-developed FLD-5B that consists of 5.4 billion comprehensive
  visual annotations on 126 million images, using an iterative strategy of
  automated image annotation and model refinement. We adopted a
  sequence-to-sequence structure to train Florence-2 to perform versatile and
  comprehensive vision tasks. Extensive evaluations on numerous tasks
  demonstrated Florence-2 to be a strong vision foundation model contender with
  unprecedented zero-shot and fine-tuning capabilities.
paper_authors._id: 65519e9fb5ec9f9a07729218
paper_authors.name: Bin Xiao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-14T09:37:54.077Z
paper_authors._id: 64ac3852692be9d12f585402
paper_authors.avatarUrl: /avatars/c018aa73d1a3d401210674222e22f1d3.svg
paper_authors.fullname: Bin Xiao
paper_authors.type: user
paper_authors.user: leoxiaobin
paper_authors._id: 65519e9fb5ec9f9a07729219
paper_authors.name: Haiping Wu
paper_authors._id: 65519e9fb5ec9f9a0772921a
paper_authors.name: Weijian Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-13T09:40:19.860Z
paper_authors._id: 6398f4b32c20654083f36cde
paper_authors.avatarUrl: /avatars/4591f514483890997c55e9e6d60bbb0f.svg
paper_authors.fullname: Weijian Xu
paper_authors.type: user
paper_authors.user: xwjabc
paper_authors._id: 65519e9fb5ec9f9a0772921b
paper_authors.name: Xiyang Dai
paper_authors._id: 65519e9fb5ec9f9a0772921c
paper_authors.name: Houdong Hu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-13T09:40:37.137Z
paper_authors._id: 62abaf9cd1f459cc044dc7b3
paper_authors.avatarUrl: /avatars/1698f5fe51e3d236fae5c71530a8218f.svg
paper_authors.fullname: Hu
paper_authors.type: user
paper_authors.user: Houdong
paper_authors._id: 65519e9fb5ec9f9a0772921d
paper_authors.name: Yumao Lu
paper_authors._id: 65519e9fb5ec9f9a0772921e
paper_authors.name: Michael Zeng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-13T09:41:54.472Z
paper_authors._id: 63fc0a7c0aa18292d5c647b6
paper_authors.avatarUrl: /avatars/4f4d50990c79f2bcebcd066b2149d20c.svg
paper_authors.fullname: Zeng
paper_authors.type: user
paper_authors.user: MichaelZeng
paper_authors._id: 65519e9fb5ec9f9a0772921f
paper_authors.name: Ce Liu
paper_authors._id: 65519e9fb5ec9f9a07729220
paper_authors.name: Lu Yuan
paper_title: Florence-2: Advancing a Unified Representation for a Variety of Vision
    Tasks
paper_summary: We introduce Florence-2, a novel vision foundation model with a unified,
  prompt-based representation for a variety of computer vision and
  vision-language tasks. While existing large vision models excel in transfer
  learning, they struggle to perform a diversity of tasks with simple
  instructions, a capability that implies handling the complexity of various
  spatial hierarchy and semantic granularity. Florence-2 was designed to take
  text-prompt as task instructions and generate desirable results in text forms,
  whether it be captioning, object detection, grounding or segmentation. This
  multi-task learning setup demands large-scale, high-quality annotated data. To
  this end, we co-developed FLD-5B that consists of 5.4 billion comprehensive
  visual annotations on 126 million images, using an iterative strategy of
  automated image annotation and model refinement. We adopted a
  sequence-to-sequence structure to train Florence-2 to perform versatile and
  comprehensive vision tasks. Extensive evaluations on numerous tasks
  demonstrated Florence-2 to be a strong vision foundation model contender with
  unprecedented zero-shot and fine-tuning capabilities.
paper_ai_keywords: vision foundation model prompt-based representation computer vision vision-language tasks transfer learning text-prompt captioning object detection grounding segmentation multi-task learning large-scale, high-quality annotated data sequence-to-sequence structure zero-shot fine-tuning