Disentangled Spatial Attention. To measure the effect of disentangled spatial attention on cross-modal interactions,
we train the models by setting the λ hyperparameter in Eq 6 to 0 or 1 . Table 7 enumerates the attention combinations,
and the results suggest that keeping only the spatial-to-spatial interaction (i.e. λs,s = 1) yields the highest NTP
accuracy. The performance differences among other configurations, such as text-to-spatial and spatial-to-text, are subtle.
Notably, the vanilla text-only self-attention mechanism yields the lowest NTP accuracy, underlining the importance
of incorporating spatial features for understanding documents with rich layouts. For all experiments in Section 4, we
therefore set λs,s = 1, λs,t = 0, and λt,s = 0. We opt for simplicity by choosing a hard mode over a soft one while
acknowledging the potential advantage of flexibility for the latter.
DOCLLM