Open-Vocabulary Scene Text Recognition via Pseudo-Image Labeling and Margin Loss
<|reference_start|>Open-Vocabulary Scene Text Recognition via Pseudo-Image Labeling and Margin Loss: Scene text recognition is an important and challenging task in computer vision. However, most prior works focus on recognizing pre-defined words, while there are various out-of-vocabulary (OOV) words in real-world applications.   In this paper, we propose a novel open-vocabulary text recognition framework, Pseudo-OCR, to recognize OOV words. The key challenge in this task is the lack of OOV training data. To solve this problem, we first propose a pseudo label generation module that leverages character detection and image inpainting to produce substantial pseudo OOV training data from real-world images. Unlike previous synthetic data, our pseudo OOV data contains real characters and backgrounds to simulate real-world applications. Secondly, to reduce noises in pseudo data, we present a semantic checking mechanism to filter semantically meaningful data. Thirdly, we introduce a quality-aware margin loss to boost the training with pseudo data. Our loss includes a margin-based part to enhance the classification ability, and a quality-aware part to penalize low-quality samples in both real and pseudo data.   Extensive experiments demonstrate that our approach outperforms the state-of-the-art on eight datasets and achieves the first rank in the ICDAR2022 challenge.<|reference_end|>
@article{ren2024open-vocabulary,
    title={Open-Vocabulary Scene Text Recognition via Pseudo-Image Labeling and
  Margin Loss},
    author={Xuhua Ren, Hengcan Shi, Jin Li},
    journal={arXiv preprint arXiv:2403.07518},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.07518},
    primaryClass={cs.CV}
}