  Dense Retrieval (DR) reaches state-of-the-art results in first-stage
retrieval, but little is known about the mechanisms that contribute to its
success. Therefore, in this work, we conduct an interpretation study of
recently proposed DR models. Specifically, we first discretize the embeddings
output by the document and query encoders. Based on the discrete
representations, we analyze the attribution of input tokens. Both qualitative
and quantitative experiments are carried out on public test collections.
Results suggest that DR models pay attention to different aspects of input and
extract various high-level topic representations. Therefore, we can regard the
representations learned by DR models as a mixture of high-level topics.

Information Retrieval