  In this paper, we propose a multi-modal search engine for interior design
that combines visual and textual queries. The goal of our engine is to retrieve
interior objects, e.g. furniture or wall clocks, that share visual and
aesthetic similarities with the query. Our search engine allows the user to
take a photo of a room and retrieve with a high recall a list of items
identical or visually similar to those present in the photo. Additionally, it
allows to return other items that aesthetically and stylistically fit well
together. To achieve this goal, our system blends the results obtained using
textual and visual modalities. Thanks to this blending strategy, we increase
the average style similarity score of the retrieved items by 11%. Our work is
implemented as a Web-based application and it is planned to be opened to the
public.

Computer Vision and Pattern Recognition