Provided are a multi-step self-attention cross-media retrieval method based on restricted text space and a retrieval system, first constructing the restricted text space of the relatively fixed vocabulary, then converting the unrestricted text space into the restricted text space, comprising: extracting the image features and text features of the restricted text space through a feature extraction network, the features comprising the global features, the regional feature set and the associated features; sending the extracted features into a feature mapping network, and extracting object-level sharing information between the image and the text through the multi-step self-attention mechanism, then collecting the useful information at each moment through a similarity measurement network to measure the similarity between the image and the text, and calculating the triple loss function; therefore, the multi-step self-attention cross-media retrieval based on the restricted text space is realized. The method and the system greatly improve the cross-media retrieval recall rate by introducing the multi-step self-attention mechanism and the associated features.
MULTI-STEP SELF-ATTENTION CROSS-MEDIA RETRIEVAL METHOD BASED ON RESTRICTED TEXT SPACE AND SYSTEM