title: R-MAE: Regions Meet Masked Autoencoders
summary: Vision-specific concepts such as "region" have played a key role in extending
  general machine learning frameworks to tasks like object detection. Given the
  success of region-based detectors for supervised learning and the progress of
  intra-image methods for contrastive learning, we explore the use of regions for
  reconstructive pre-training. Starting from Masked Autoencoding (MAE) both as a
  baseline and an inspiration, we propose a parallel pre-text task tailored to
  address the one-to-many mapping between images and regions. Since such regions
  can be generated in an unsupervised way, our approach (R-MAE) inherits the wide
  applicability from MAE, while being more "region-aware". We conduct thorough
  analyses during the development of R-MAE, and converge on a variant that is
  both effective and efficient (1.3% overhead over MAE). Moreover, it shows
  consistent quantitative improvements when generalized to various pre-training
  data and downstream detection and segmentation benchmarks. Finally, we provide
  extensive qualitative visualizations to enhance the understanding of R-MAE's
  behaviour and potential. Code will be made available at
  https://github.com/facebookresearch/r-mae.
paper_authors._id: 6482c107474d4d463d4eee53
paper_authors.name: Duy-Kien Nguyen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-12T13:09:02.154Z
paper_authors._id: 6317dfa31dd5a319a67f2556
paper_authors.avatarUrl: /avatars/4a6b5748b8d95a15be560af5fd2de998.svg
paper_authors.fullname: Kien Nguyen
paper_authors.type: user
paper_authors.user: duykien
paper_authors._id: 6482c107474d4d463d4eee54
paper_authors.name: Vaibhav Aggarwal
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-12T13:10:14.815Z
paper_authors._id: 629f97b73fa7e9d614e92212
paper_authors.avatarUrl: /avatars/6903732275dc71213cfbf82a98c39084.svg
paper_authors.fullname: Vaibhav Aggarwal
paper_authors.type: user
paper_authors.user: vaibhavagg303
paper_authors._id: 6482c107474d4d463d4eee55
paper_authors.name: Yanghao Li
paper_authors._id: 6482c107474d4d463d4eee56
paper_authors.name: Martin R. Oswald
paper_authors._id: 6482c107474d4d463d4eee57
paper_authors.name: Alexander Kirillov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-12T13:11:21.278Z
paper_authors._id: 630fa75802ce39336c45f31c
paper_authors.avatarUrl: /avatars/0e4e3839a4e6f182c2618eb35acef29a.svg
paper_authors.fullname: Alexander Kirillov
paper_authors.type: user
paper_authors.user: akirillov
paper_authors._id: 6482c107474d4d463d4eee58
paper_authors.name: Cees G. M. Snoek
paper_authors._id: 6482c107474d4d463d4eee59
paper_authors.name: Xinlei Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-12T09:11:53.653Z
paper_authors._id: 63e58e3a006a775275e59e41
paper_authors.avatarUrl: /avatars/75262a35b27a2ae1939df9118120d99e.svg
paper_authors.fullname: Xinlei Chen
paper_authors.type: user
paper_authors.user: endernewton
paper_title: R-MAE: Regions Meet Masked Autoencoders
paper_summary: Vision-specific concepts such as "region" have played a key role in extending
  general machine learning frameworks to tasks like object detection. Given the
  success of region-based detectors for supervised learning and the progress of
  intra-image methods for contrastive learning, we explore the use of regions for
  reconstructive pre-training. Starting from Masked Autoencoding (MAE) both as a
  baseline and an inspiration, we propose a parallel pre-text task tailored to
  address the one-to-many mapping between images and regions. Since such regions
  can be generated in an unsupervised way, our approach (R-MAE) inherits the wide
  applicability from MAE, while being more "region-aware". We conduct thorough
  analyses during the development of R-MAE, and converge on a variant that is
  both effective and efficient (1.3% overhead over MAE). Moreover, it shows
  consistent quantitative improvements when generalized to various pre-training
  data and downstream detection and segmentation benchmarks. Finally, we provide
  extensive qualitative visualizations to enhance the understanding of R-MAE's
  behaviour and potential. Code will be made available at
  https://github.com/facebookresearch/r-mae.
paper_ai_keywords: Masked Autoencoding (MAE) region-based detectors contrastive learning reconstructive pre-training one-to-many mapping unsupervised R-MAE pre-training data downstream detection segmentation benchmarks qualitative visualizations