title: CostFormer:Cost Transformer for Cost Aggregation in Multi-view Stereo
summary: The core of Multi-view Stereo(MVS) is the matching process among reference
  and source pixels. Cost aggregation plays a significant role in this process,
  while previous methods focus on handling it via CNNs. This may inherit the
  natural limitation of CNNs that fail to discriminate repetitive or incorrect
  matches due to limited local receptive fields. To handle the issue, we aim to
  involve Transformer into cost aggregation. However, another problem may occur
  due to the quadratically growing computational complexity caused by
  Transformer, resulting in memory overflow and inference latency. In this paper,
  we overcome these limits with an efficient Transformer-based cost aggregation
  network, namely CostFormer. The Residual Depth-Aware Cost Transformer(RDACT) is
  proposed to aggregate long-range features on cost volume via self-attention
  mechanisms along the depth and spatial dimensions. Furthermore, Residual
  Regression Transformer(RRT) is proposed to enhance spatial attention. The
  proposed method is a universal plug-in to improve learning-based MVS methods.
paper_authors._id: 6465914e3732b3510ecc0c8e
paper_authors.name: Weitao Chen
paper_authors._id: 6465914e3732b3510ecc0c8f
paper_authors.name: Hongbin Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-26T09:14:08.712Z
paper_authors._id: 6395879d1da3832bd466fcc4
paper_authors.avatarUrl: /avatars/091d22fb522d3c8a46d56e835d239272.svg
paper_authors.fullname: Hongbin Xu
paper_authors.type: user
paper_authors.user: ToughStoneX
paper_authors._id: 6465914e3732b3510ecc0c90
paper_authors.name: Zhipeng Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-26T09:14:27.955Z
paper_authors._id: 642bd96e4d7a550711e83a18
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/642bd96e4d7a550711e83a18/Rn7LCnXqGiMWCAxiTyKTz.jpeg
paper_authors.fullname: zhipeng zhou
paper_authors.type: user
paper_authors.user: randy-zhou
paper_authors._id: 6465914e3732b3510ecc0c91
paper_authors.name: Yang Liu
paper_authors._id: 6465914e3732b3510ecc0c92
paper_authors.name: Baigui Sun
paper_authors._id: 6465914e3732b3510ecc0c93
paper_authors.name: Wenxiong Kang
paper_authors._id: 6465914e3732b3510ecc0c94
paper_authors.name: Xuansong Xie
paper_title: CostFormer:Cost Transformer for Cost Aggregation in Multi-view Stereo
paper_summary: The core of Multi-view Stereo(MVS) is the matching process among reference
  and source pixels. Cost aggregation plays a significant role in this process,
  while previous methods focus on handling it via CNNs. This may inherit the
  natural limitation of CNNs that fail to discriminate repetitive or incorrect
  matches due to limited local receptive fields. To handle the issue, we aim to
  involve Transformer into cost aggregation. However, another problem may occur
  due to the quadratically growing computational complexity caused by
  Transformer, resulting in memory overflow and inference latency. In this paper,
  we overcome these limits with an efficient Transformer-based cost aggregation
  network, namely CostFormer. The Residual Depth-Aware Cost Transformer(RDACT) is
  proposed to aggregate long-range features on cost volume via self-attention
  mechanisms along the depth and spatial dimensions. Furthermore, Residual
  Regression Transformer(RRT) is proposed to enhance spatial attention. The
  proposed method is a universal plug-in to improve learning-based MVS methods.
paper_ai_keywords: Multi-view Stereo (MVS) cost aggregation CNNs natural limitation of CNNs local receptive fields Transformer computational complexity memory overflow inference latency CostFormer Multi-view Stereo (MVS) cost aggregation CNNs natural limitation of CNNs local receptive fields Transformer computational complexity memory overflow inference latency CostFormer Residual Depth-AWARE Cost Transformer (RDACT) self-attention mechanisms depth and spatial dimensions Residual Regression Transformer (RRT) spatial attention