images). In supervised instruction tuning, we can adhere to the established practices used in other works, based on
desired outputs such as text or images.
The main concept for a cohesive block is to ensure meaningful infilling during the pre-training phase, preventing
disconnected predictions. However, the choice of OCR engines to obtain such cohesive blocks remains an open area
for exploration. Practical comparisons with various OCR engines and/or layout parsers are left as future work, as
LayoutLMs underscore the importance of accurate OCR for improved VQA results. They leverage the Microsoft
Azure API, demonstrating superior performance compared to TesseractOCR, as indicated in the DocVQA leaderboard.6
Consequently, researchers are also encouraged to utilize more accurate OCR engines for potential enhancements, if
such resources are available.
We have presented a collection of SDDS results alongside zero-shot outcomes. To mitigate prompt influence in the
DOCLLM