Hi Jung, thank you for your reply.
Indeed DPR uses cross-attention in the reader, however, the use case here is a bit different. In other words, at the end of the cross-encoder, I need a similarity value of some sort, which is not the case at the Reader.
I have some updates. After more investigation, the authors of Poly-encoders: architectures and pre-training strategies for fast and accurate multi-sentence scoring 3 give a high-level overview of how they obtain the similarity value with the Cross-Encoder in the section 4.3. Furthermore, there are more insights on the ParlAI transformers repo 6, where they say: transformer/crossencoder: A retrieval-based agent that jointly encodes a context and candidate sequence in a single BERT-based 2 Transformer, with a final linear layer used to compute a score. A candidate is chosen via the highest-scoring encoding.
It seems that I can use this approach, we can obtain scores (not normalized) among all the pairs and then use a loss function like NLL or cross-entropy.