readme: ---
  language: multilingual
  datasets:
  - NQ
  - Trivia
  - SQuAD
  - MLQA
  - DRCD
  ---
  
  # dpr-ctx_encoder-bert-base-multilingual
  
  ## Description
  
  Multilingual DPR Model base on bert-base-multilingual-cased. 
  [DPR model](https://arxiv.org/abs/2004.04906)
  [DPR repo](https://github.com/facebookresearch/DPR)
  
  ## Data
  1. [NQ](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py)
  2. [Trivia](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py)
  3. [SQuAD](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py)
  4. [DRCD*](https://github.com/DRCKnowledgeTeam/DRCD)
  5. [MLQA*](https://github.com/facebookresearch/MLQA)
  
  `question pairs for train`： 644,217  
  `question pairs for dev`： 73,710
  
  *DRCD and MLQA are converted using script from haystack [squad_to_dpr.py](https://github.com/deepset-ai/haystack/blob/master/haystack/retriever/squad_to_dpr.py)
  
  ## Training Script
  I use the script from [haystack](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial9_DPR_training.ipynb)
  
  ## Usage
  
  ```python
  from transformers import DPRContextEncoder, DPRContextEncoderTokenizer
  tokenizer = DPRContextEncoderTokenizer.from_pretrained('voidful/dpr-ctx_encoder-bert-base-multilingual')
  model = DPRContextEncoder.from_pretrained('voidful/dpr-ctx_encoder-bert-base-multilingual')
  input_ids = tokenizer("Hello, is my dog cute ?", return_tensors='pt')["input_ids"]
  embeddings = model(input_ids).pooler_output
  ```
  
  Follow the tutorial from `haystack`:
  [Better Retrievers via "Dense Passage Retrieval"](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial6_Better_Retrieval_via_DPR.ipynb)
  ```
  from haystack.retriever.dense import DensePassageRetriever
  retriever = DensePassageRetriever(document_store=document_store,
                                    query_embedding_model="voidful/dpr-question_encoder-bert-base-multilingual",
                                    passage_embedding_model="voidful/dpr-ctx_encoder-bert-base-multilingual",
                                    max_seq_len_query=64,
                                    max_seq_len_passage=256,
                                    batch_size=16,
                                    use_gpu=True,
                                    embed_title=True,
                                    use_fast_tokenizers=True)
  ```
  