The invention discloses a cross-modal vision and text information matching method and device. The method comprises the following steps: detecting blocks of an important object scene in a picture through a mature target recognition system, the total number of the blocks being X; the visual vector information of the blocks is input into a visual encoder, so that the relation and the comprehensive information among the blocks are constructed, the visual vector information hx containing the context is obtained, and x belongs to X; for text information with the length of Y, encoding by adopting a pre-trained text encoder model to obtain a context embedding vector hy of each word, y belonging to Y; performing matching calculation on all hx and hy pairs to obtain X * Y scores; respectively extracting the maximum scores of each column and each row for the x axis and the y axis, and carrying out nonlinear transformation on the scores; the final score s is the matching degree, and the final score s is the average of all g'.
Cross-modal vision and text information matching method and device