Florence-2: Advancing a Unified Representation for a Variety of Vision
  Tasks
We introduce Florence-2, a novel vision foundation model with a unified,
prompt-based representation for a variety of computer vision and
vision-language tasks. While existing large vision models excel in transfer
learning, they struggle to perform a diversity of tasks with simple
instructions, a capability that implies handling the complexity of various
spatial hierarchy and semantic granularity. Florence-2 was designed to take
text-prompt as task instructions and generate desirable results in text forms,
whether it be captioning, object detection, grounding or segmentation. This
multi-task learning setup demands large-scale, high-quality annotated data. To
this end, we co-developed FLD-5B that consists of 5.4 billion comprehensive
visual annotations on 126 million images, using an iterative strategy of
automated image annotation and model refinement. We adopted a
sequence-to-sequence structure to train Florence-2 to perform versatile and
comprehensive vision tasks. Extensive evaluations on numerous tasks
demonstrated Florence-2 to be a strong vision foundation model contender with
unprecedented zero-shot and fine-tuning capabilities.
65519e9fb5ec9f9a07729218
Bin Xiao
claimed_verified
2023-11-14T09:37:54.077Z
64ac3852692be9d12f585402
/avatars/c018aa73d1a3d401210674222e22f1d3.svg
Bin Xiao
user
leoxiaobin
65519e9fb5ec9f9a07729219
Haiping Wu
65519e9fb5ec9f9a0772921a
Weijian Xu
admin_assigned
2023-11-13T09:40:19.860Z
6398f4b32c20654083f36cde
/avatars/4591f514483890997c55e9e6d60bbb0f.svg
Weijian Xu
user
xwjabc
65519e9fb5ec9f9a0772921b
Xiyang Dai
65519e9fb5ec9f9a0772921c
Houdong Hu
admin_assigned
2023-11-13T09:40:37.137Z
62abaf9cd1f459cc044dc7b3
/avatars/1698f5fe51e3d236fae5c71530a8218f.svg
Hu
user
Houdong
65519e9fb5ec9f9a0772921d
Yumao Lu
65519e9fb5ec9f9a0772921e
Michael Zeng
admin_assigned
2023-11-13T09:41:54.472Z
63fc0a7c0aa18292d5c647b6
/avatars/4f4d50990c79f2bcebcd066b2149d20c.svg
Zeng
user
MichaelZeng
65519e9fb5ec9f9a0772921f
Ce Liu
65519e9fb5ec9f9a07729220
Lu Yuan
Florence-2: Advancing a Unified Representation for a Variety of Vision
  Tasks
We introduce Florence-2, a novel vision foundation model with a unified,
prompt-based representation for a variety of computer vision and
vision-language tasks. While existing large vision models excel in transfer
learning, they struggle to perform a diversity of tasks with simple
instructions, a capability that implies handling the complexity of various
spatial hierarchy and semantic granularity. Florence-2 was designed to take
text-prompt as task instructions and generate desirable results in text forms,
whether it be captioning, object detection, grounding or segmentation. This
multi-task learning setup demands large-scale, high-quality annotated data. To
this end, we co-developed FLD-5B that consists of 5.4 billion comprehensive
visual annotations on 126 million images, using an iterative strategy of
automated image annotation and model refinement. We adopted a
sequence-to-sequence structure to train Florence-2 to perform versatile and
comprehensive vision tasks. Extensive evaluations on numerous tasks
demonstrated Florence-2 to be a strong vision foundation model contender with
unprecedented zero-shot and fine-tuning capabilities.
A new prompt-based vision foundation model, Florence-2, is introduced for diverse vision and vision-language tasks, achieving strong zero-shot and fine-tuning capabilities with comprehensive annotations.
prompt-based representation sequence-to-sequence structure zero-shot capabilities fine-tuning capabilities