title: Gen2Det: Generate to Detect
summary: Recently diffusion models have shown improvement in synthetic image quality
  as well as better control in generation. We motivate and present Gen2Det, a
  simple modular pipeline to create synthetic training data for object detection
  for free by leveraging state-of-the-art grounded image generation methods.
  Unlike existing works which generate individual object instances, require
  identifying foreground followed by pasting on other images, we simplify to
  directly generating scene-centric images. In addition to the synthetic data,
  Gen2Det also proposes a suite of techniques to best utilize the generated data,
  including image-level filtering, instance-level filtering, and better training
  recipe to account for imperfections in the generation. Using Gen2Det, we show
  healthy improvements on object detection and segmentation tasks under various
  settings and agnostic to detection methods. In the long-tailed detection
  setting on LVIS, Gen2Det improves the performance on rare categories by a large
  margin while also significantly improving the performance on other categories,
  e.g. we see an improvement of 2.13 Box AP and 1.84 Mask AP over just training
  on real data on LVIS with Mask R-CNN. In the low-data regime setting on COCO,
  Gen2Det consistently improves both Box and Mask AP by 2.27 and 1.85 points. In
  the most general detection setting, Gen2Det still demonstrates robust
  performance gains, e.g. it improves the Box and Mask AP on COCO by 0.45 and
  0.32 points.
paper_authors._id: 65728dfaca0139e4acc23821
paper_authors.name: Saksham Suri
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-08T10:39:38.885Z
paper_authors._id: 645417e98617183806210cfc
paper_authors.avatarUrl: /avatars/7a3ba5b0c38878db51da7091fed15f99.svg
paper_authors.fullname: Saksham Suri
paper_authors.type: user
paper_authors.user: sakshams
paper_authors._id: 65728dfaca0139e4acc23822
paper_authors.name: Fanyi Xiao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-08T10:39:47.351Z
paper_authors._id: 641cf83645487810d136cb34
paper_authors.avatarUrl: /avatars/95bfc200f86abbe9e8df0308517a9ce4.svg
paper_authors.fullname: Fanyi Xiao
paper_authors.type: user
paper_authors.user: fanyix
paper_authors._id: 65728dfaca0139e4acc23823
paper_authors.name: Animesh Sinha
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-08T10:39:53.325Z
paper_authors._id: 63772d28e069bdcdbb8e4a29
paper_authors.avatarUrl: /avatars/be71b00e009f2093a3ff79f07815e68e.svg
paper_authors.fullname: Animesh Sinha
paper_authors.type: user
paper_authors.user: animeshsinha
paper_authors._id: 65728dfaca0139e4acc23824
paper_authors.name: Sean Chang Culatana
paper_authors._id: 65728dfaca0139e4acc23825
paper_authors.name: Raghuraman Krishnamoorthi
paper_authors._id: 65728dfaca0139e4acc23826
paper_authors.name: Chenchen Zhu
paper_authors._id: 65728dfaca0139e4acc23827
paper_authors.name: Abhinav Shrivastava
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-08T10:40:43.124Z
paper_authors._id: 64b683acb20146a0d307222c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/l665j1WeROwhQKgIsszYp.jpeg
paper_authors.fullname: Abhinav Shrivastava
paper_authors.type: user
paper_authors.user: abhi2610
paper_title: Gen2Det: Generate to Detect
paper_summary: Recently diffusion models have shown improvement in synthetic image quality
  as well as better control in generation. We motivate and present Gen2Det, a
  simple modular pipeline to create synthetic training data for object detection
  for free by leveraging state-of-the-art grounded image generation methods.
  Unlike existing works which generate individual object instances, require
  identifying foreground followed by pasting on other images, we simplify to
  directly generating scene-centric images. In addition to the synthetic data,
  Gen2Det also proposes a suite of techniques to best utilize the generated data,
  including image-level filtering, instance-level filtering, and better training
  recipe to account for imperfections in the generation. Using Gen2Det, we show
  healthy improvements on object detection and segmentation tasks under various
  settings and agnostic to detection methods. In the long-tailed detection
  setting on LVIS, Gen2Det improves the performance on rare categories by a large
  margin while also significantly improving the performance on other categories,
  e.g. we see an improvement of 2.13 Box AP and 1.84 Mask AP over just training
  on real data on LVIS with Mask R-CNN. In the low-data regime setting on COCO,
  Gen2Det consistently improves both Box and Mask AP by 2.27 and 1.85 points. In
  the most general detection setting, Gen2Det still demonstrates robust
  performance gains, e.g. it improves the Box and Mask AP on COCO by 0.45 and
  0.32 points.
paper_ai_summary: Gen2Det generates scene-centric synthetic training data for object detection using grounded image generation and improves performance across various settings.
paper_ai_keywords: diffusion models scene-centric images image-level filtering instance-level filtering training recipe long-tailed detection low-data regime Box AP Mask AP LVIS Mask R-CNN COCO