LAIP: Learning Local Alignment from Image-Phrase Modeling for Text-based Person Search
<|reference_start|>LAIP: Learning Local Alignment from Image-Phrase Modeling for Text-based Person Search: Text-based person search aims at retrieving images of a particular person based on a given textual description. A common solution for this task is to directly match the entire images and texts, i.e., global alignment, which fails to deal with discerning specific details that discriminate against appearance-similar people. As a result, some works shift their attention towards local alignment. One group matches fine-grained parts using forward attention weights of the transformer yet underutilizes information. Another implicitly conducts local alignment by reconstructing masked parts based on unmasked context yet with a biased masking strategy. All limit performance improvement. This paper proposes the Local Alignment from Image-Phrase modeling (LAIP) framework, with Bidirectional Attention-weighted local alignment (BidirAtt) and Mask Phrase Modeling (MPM) module.BidirAtt goes beyond the typical forward attention by considering the gradient of the transformer as backward attention, utilizing two-sided information for local alignment. MPM focuses on mask reconstruction within the noun phrase rather than the entire text, ensuring an unbiased masking strategy. Extensive experiments conducted on the CUHK-PEDES, ICFG-PEDES, and RSTPReid datasets demonstrate the superiority of the LAIP framework over existing methods.<|reference_end|>
@article{wang2024laip:,
    title={LAIP: Learning Local Alignment from Image-Phrase Modeling for Text-based
  Person Search},
    author={Haiguang Wang, Yu Wu, Mengxia Wu, Cao Min, Min Zhang},
    journal={arXiv preprint arXiv:2406.10845},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.10845},
    primaryClass={cs.CV}
}