Multimodal Point-of-Interest Recommendation
<|reference_start|>Multimodal Point-of-Interest Recommendation: Large Language Models are applied to recommendation tasks such as items to buy and news articles to read. Point of Interest is quite a new area to sequential recommendation based on language representations of multimodal datasets. As a first step to prove our concepts, we focused on restaurant recommendation based on each user's past visit history. When choosing a next restaurant to visit, a user would consider genre and location of the venue and, if available, pictures of dishes served there. We created a pseudo restaurant check-in history dataset from the Foursquare dataset and the FoodX-251 dataset by converting pictures into text descriptions with a multimodal model called LLaVA, and used a language-based sequential recommendation framework named Recformer proposed in 2023. A model trained on this semi-multimodal dataset has outperformed another model trained on the same dataset without picture descriptions. This suggests that this semi-multimodal model reflects actual human behaviours and that our path to a multimodal recommendation model is in the right direction.<|reference_end|>
@article{kanzawa2024multimodal,
    title={Multimodal Point-of-Interest Recommendation},
    author={Yuta Kanzawa, Toyotaro Suzumura, Hiroki Kanezashi, Jiawei Yong,
  Shintaro Fukushima},
    journal={arXiv preprint arXiv:2410.03265},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.03265},
    primaryClass={cs.IR cs.SI}
}