  Accurate measurement of the offset from roof-to-footprint in
very-high-resolution remote sensing imagery is crucial for urban information
extraction tasks. With the help of deep learning, existing methods typically
rely on two-stage CNN models to extract regions of interest on building feature
maps. At the first stage, a Region Proposal Network (RPN) is applied to extract
thousands of ROIs (Region of Interests) which will post-imported into a
Region-based Convolutional Neural Networks (RCNN) to extract wanted
information. However, because of inflexible RPN, these methods often lack
effective user interaction, encounter difficulties in instance correspondence,
and struggle to keep up with the advancements in general artificial
intelligence. This paper introduces an interactive Transformer model combined
with a prompt encoder to precisely extract building segmentation as well as the
offset vectors from roofs to footprints. In our model, a powerful module,
namely ROAM, was tailored for common problems in predicting roof-to-footprint
offsets. We tested our model's feasibility on the publicly available BONAI
dataset, achieving a significant reduction in Prompt-Instance-Level offset
errors ranging from 14.6% to 16.3%. Additionally, we developed a Distance-NMS
algorithm tailored for large-scale building offsets, significantly enhancing
the accuracy of predicted building offset angles and lengths in a
straightforward and efficient manner. To further validate the model's
robustness, we created a new test set using 0.5m remote sensing imagery from
Huizhou, China, for inference testing. Our code, training methods, and the
updated dataset will be accessable at https://github.com/likaiucas.

Computer Vision and Pattern Recognition