title: MaskRIS: Semantic Distortion-aware Data Augmentation for Referring Image
    Segmentation
summary: Referring Image Segmentation (RIS) is an advanced vision-language task that
  involves identifying and segmenting objects within an image as described by
  free-form text descriptions. While previous studies focused on aligning visual
  and language features, exploring training techniques, such as data
  augmentation, remains underexplored. In this work, we explore effective data
  augmentation for RIS and propose a novel training framework called Masked
  Referring Image Segmentation (MaskRIS). We observe that the conventional image
  augmentations fall short of RIS, leading to performance degradation, while
  simple random masking significantly enhances the performance of RIS. MaskRIS
  uses both image and text masking, followed by Distortion-aware Contextual
  Learning (DCL) to fully exploit the benefits of the masking strategy. This
  approach can improve the model's robustness to occlusions, incomplete
  information, and various linguistic complexities, resulting in a significant
  performance improvement. Experiments demonstrate that MaskRIS can easily be
  applied to various RIS models, outperforming existing methods in both fully
  supervised and weakly supervised settings. Finally, MaskRIS achieves new
  state-of-the-art performance on RefCOCO, RefCOCO+, and RefCOCOg datasets. Code
  is available at https://github.com/naver-ai/maskris.
paper_authors._id: 674fdf3e6bc2d6af0e165bf0
paper_authors.name: Minhyun Lee
paper_authors._id: 674fdf3e6bc2d6af0e165bf1
paper_authors.name: Seungho Lee
paper_authors._id: 674fdf3e6bc2d6af0e165bf2
paper_authors.name: Song Park
paper_authors._id: 674fdf3e6bc2d6af0e165bf3
paper_authors.name: Dongyoon Han
paper_authors._id: 674fdf3e6bc2d6af0e165bf4
paper_authors.name: Byeongho Heo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-04T09:44:21.729Z
paper_authors._id: 64b9feed96676e40d0fa89a7
paper_authors.avatarUrl: /avatars/2154a6ceb87677ad2c9d9620de5b18ec.svg
paper_authors.fullname: Byeongho Heo
paper_authors.type: user
paper_authors.user: bhheo
paper_authors._id: 674fdf3e6bc2d6af0e165bf5
paper_authors.name: Hyunjung Shim
paper_title: MaskRIS: Semantic Distortion-aware Data Augmentation for Referring Image
    Segmentation
paper_summary: Referring Image Segmentation (RIS) is an advanced vision-language task that
  involves identifying and segmenting objects within an image as described by
  free-form text descriptions. While previous studies focused on aligning visual
  and language features, exploring training techniques, such as data
  augmentation, remains underexplored. In this work, we explore effective data
  augmentation for RIS and propose a novel training framework called Masked
  Referring Image Segmentation (MaskRIS). We observe that the conventional image
  augmentations fall short of RIS, leading to performance degradation, while
  simple random masking significantly enhances the performance of RIS. MaskRIS
  uses both image and text masking, followed by Distortion-aware Contextual
  Learning (DCL) to fully exploit the benefits of the masking strategy. This
  approach can improve the model's robustness to occlusions, incomplete
  information, and various linguistic complexities, resulting in a significant
  performance improvement. Experiments demonstrate that MaskRIS can easily be
  applied to various RIS models, outperforming existing methods in both fully
  supervised and weakly supervised settings. Finally, MaskRIS achieves new
  state-of-the-art performance on RefCOCO, RefCOCO+, and RefCOCOg datasets. Code
  is available at https://github.com/naver-ai/maskris.
paper_ai_summary: A novel training framework called Masked Referring Image Segmentation (MaskRIS) enhances RIS performance through image and text masking combined with Distortion-aware Contextual Learning.
paper_ai_keywords: referring image segmentation Masked Referring Image Segmentation MaskRIS data augmentation image masking text masking Distortion-aware Contextual Learning RIS RefCOCO RefCOCO+ RefCOCOg