CostFormer:Cost Transformer for Cost Aggregation in Multi-view Stereo
The core of Multi-view Stereo(MVS) is the matching process among reference
and source pixels. Cost aggregation plays a significant role in this process,
while previous methods focus on handling it via CNNs. This may inherit the
natural limitation of CNNs that fail to discriminate repetitive or incorrect
matches due to limited local receptive fields. To handle the issue, we aim to
involve Transformer into cost aggregation. However, another problem may occur
due to the quadratically growing computational complexity caused by
Transformer, resulting in memory overflow and inference latency. In this paper,
we overcome these limits with an efficient Transformer-based cost aggregation
network, namely CostFormer. The Residual Depth-Aware Cost Transformer(RDACT) is
proposed to aggregate long-range features on cost volume via self-attention
mechanisms along the depth and spatial dimensions. Furthermore, Residual
Regression Transformer(RRT) is proposed to enhance spatial attention. The
proposed method is a universal plug-in to improve learning-based MVS methods.
6465914e3732b3510ecc0c8e
Weitao Chen
6465914e3732b3510ecc0c8f
Hongbin Xu
admin_assigned
2023-05-26T09:14:08.712Z
6395879d1da3832bd466fcc4
/avatars/091d22fb522d3c8a46d56e835d239272.svg
Hongbin Xu
user
ToughStoneX
6465914e3732b3510ecc0c90
Zhipeng Zhou
admin_assigned
2023-05-26T09:14:27.955Z
642bd96e4d7a550711e83a18
https://cdn-avatars.huggingface.co/v1/production/uploads/642bd96e4d7a550711e83a18/Rn7LCnXqGiMWCAxiTyKTz.jpeg
zhipeng zhou
user
randy-zhou
6465914e3732b3510ecc0c91
Yang Liu
6465914e3732b3510ecc0c92
Baigui Sun
6465914e3732b3510ecc0c93
Wenxiong Kang
6465914e3732b3510ecc0c94
Xuansong Xie
CostFormer:Cost Transformer for Cost Aggregation in Multi-view Stereo
The core of Multi-view Stereo(MVS) is the matching process among reference
and source pixels. Cost aggregation plays a significant role in this process,
while previous methods focus on handling it via CNNs. This may inherit the
natural limitation of CNNs that fail to discriminate repetitive or incorrect
matches due to limited local receptive fields. To handle the issue, we aim to
involve Transformer into cost aggregation. However, another problem may occur
due to the quadratically growing computational complexity caused by
Transformer, resulting in memory overflow and inference latency. In this paper,
we overcome these limits with an efficient Transformer-based cost aggregation
network, namely CostFormer. The Residual Depth-Aware Cost Transformer(RDACT) is
proposed to aggregate long-range features on cost volume via self-attention
mechanisms along the depth and spatial dimensions. Furthermore, Residual
Regression Transformer(RRT) is proposed to enhance spatial attention. The
proposed method is a universal plug-in to improve learning-based MVS methods.
The proposed CostFormer network uses Transformers to efficiently perform cost aggregation in Multi-view Stereo, addressing limitations of CNNs with residual depth-aware and regression-based self-attention mechanisms.
Multi-view Stereo MVS cost aggregation CNNs Transformers self-attention Residual Depth-Aware Cost Transformer RDACT Residual Regression Transformer RRT