Hello everyone,
I created my own MSMARCO dataset with these two tutorials: Writing a dataset loading script 3 and squad dataset loading script 1. Furthermore, I added the FAISS index as Adding a FAISS index 7. I have generated context embeddings with DPRContextEncoder, DPRContextEncoderTokenizer and question embeddings with DPRQuestionEncoder and DPRQuestionEncoderTokenizer. Thus the embedding vectors have a size of 768.
I came to realize that something was not making much sense. FAISS is a library for efficient similarity search and clustering of dense vectors. Thus, I would expect that getting the top-k nearest examples would not be a “very long task”. Therefore, I have done some tests on running get_nearest_examples() with a FAISS index over two different queries (4000 and 4001) several times on the CPU.
attempts12832×263 33.2 KB
attempts345818×385 49.6 KB
As you can observe, the first time I try to retrieve the top-1000 passages for both queries 4000 and 4001 the wall time is ~9-10min. After that, the wall time matches the CPU time, which seems to indicate that get_nearest_examples() is I/O bounded and uses efficient caching.
Should I be expecting faster retrievals on the first attempt? Notice that the problem after all is not on the computation (it seems to be no advantages on moving the index to the GPU, as I tried on the above replies). All the CPU times “were short”, which makes sense since FAISS is a library for efficient similarity search and clustering of dense vectors.