title: Rethinking RGB-Event Semantic Segmentation with a Novel Bidirectional
    Motion-enhanced Event Representation
summary: Event cameras capture motion dynamics, offering a unique modality with great
  potential in various computer vision tasks. However, RGB-Event fusion faces
  three intrinsic misalignments: (i) temporal, (ii) spatial, and (iii) modal
  misalignment. Existing voxel grid representations neglect temporal correlations
  between consecutive event windows, and their formulation with simple
  accumulation of asynchronous and sparse events is incompatible with the
  synchronous and dense nature of RGB modality. To tackle these challenges, we
  propose a novel event representation, Motion-enhanced Event Tensor (MET), which
  transforms sparse event voxels into a dense and temporally coherent form by
  leveraging dense optical flows and event temporal features. In addition, we
  introduce a Frequency-aware Bidirectional Flow Aggregation Module (BFAM) and a
  Temporal Fusion Module (TFM). BFAM leverages the frequency domain and MET to
  mitigate modal misalignment, while bidirectional flow aggregation and temporal
  fusion mechanisms resolve spatiotemporal misalignment. Experimental results on
  two large-scale datasets demonstrate that our framework significantly
  outperforms state-of-the-art RGB-Event semantic segmentation approaches. Our
  code is available at: https://github.com/zyaocoder/BRENet.
paper_authors._id: 681a662d26c8bff22d483fac
paper_authors.name: Zhen Yao
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-06T19:44:04.570Z
paper_authors._id: 654939b935fa2ade2defa15b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/654939b935fa2ade2defa15b/r-Ku79eVKxeCa1f8Ogf9C.png
paper_authors.fullname: Zhen Yao
paper_authors.type: user
paper_authors.user: Chrisathy
paper_authors._id: 681a662d26c8bff22d483fad
paper_authors.name: Xiaowen Ying
paper_authors._id: 681a662d26c8bff22d483fae
paper_authors.name: Mooi Choo Chuah
paper_title: Rethinking RGB-Event Semantic Segmentation with a Novel Bidirectional
    Motion-enhanced Event Representation
paper_summary: Event cameras capture motion dynamics, offering a unique modality with great
  potential in various computer vision tasks. However, RGB-Event fusion faces
  three intrinsic misalignments: (i) temporal, (ii) spatial, and (iii) modal
  misalignment. Existing voxel grid representations neglect temporal correlations
  between consecutive event windows, and their formulation with simple
  accumulation of asynchronous and sparse events is incompatible with the
  synchronous and dense nature of RGB modality. To tackle these challenges, we
  propose a novel event representation, Motion-enhanced Event Tensor (MET), which
  transforms sparse event voxels into a dense and temporally coherent form by
  leveraging dense optical flows and event temporal features. In addition, we
  introduce a Frequency-aware Bidirectional Flow Aggregation Module (BFAM) and a
  Temporal Fusion Module (TFM). BFAM leverages the frequency domain and MET to
  mitigate modal misalignment, while bidirectional flow aggregation and temporal
  fusion mechanisms resolve spatiotemporal misalignment. Experimental results on
  two large-scale datasets demonstrate that our framework significantly
  outperforms state-of-the-art RGB-Event semantic segmentation approaches. Our
  code is available at: https://github.com/zyaocoder/BRENet.
paper_ai_summary: A novel event representation and fusion framework enhances RGB-Event semantic segmentation by addressing temporal, spatial, and modal misalignments using dense optical flows and bidirectional flow aggregation.
paper_ai_keywords: voxel grid representations Motion-enhanced Event Tensor (MET) Frequency-aware Bidirectional Flow Aggregation Module (BFAM) Temporal Fusion Module (TFM) semantic segmentation dense optical flows bidirectional flow aggregation temporal fusion mechanisms