radu.mutilica commited on
Commit
b352fd9
·
1 Parent(s): a6fe247

modified readme

Browse files
Files changed (1) hide show
  1. README.md +3 -60
README.md CHANGED
@@ -1,64 +1,7 @@
1
  ---
2
  license: apache-2.0
3
  ---
4
- # Cross-Encoder for MS Marco
 
5
 
6
- This model was trained on the [MS Marco Passage Ranking](https://github.com/microsoft/MSMARCO-Passage-Ranking) task.
7
-
8
- The model can be used for Information Retrieval: Given a query, encode the query will all possible passages (e.g. retrieved with ElasticSearch). Then sort the passages in a decreasing order. See [SBERT.net Retrieve & Re-rank](https://www.sbert.net/examples/applications/retrieve_rerank/README.html) for more details. The training code is available here: [SBERT.net Training MS Marco](https://github.com/UKPLab/sentence-transformers/tree/master/examples/training/ms_marco)
9
-
10
-
11
- ## Usage with Transformers
12
-
13
- ```python
14
- from transformers import AutoTokenizer, AutoModelForSequenceClassification
15
- import torch
16
-
17
- model = AutoModelForSequenceClassification.from_pretrained('model_name')
18
- tokenizer = AutoTokenizer.from_pretrained('model_name')
19
-
20
- features = tokenizer(['How many people live in Berlin?', 'How many people live in Berlin?'], ['Berlin has a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.', 'New York City is famous for the Metropolitan Museum of Art.'], padding=True, truncation=True, return_tensors="pt")
21
-
22
- model.eval()
23
- with torch.no_grad():
24
- scores = model(**features).logits
25
- print(scores)
26
- ```
27
-
28
-
29
- ## Usage with SentenceTransformers
30
-
31
- The usage becomes easier when you have [SentenceTransformers](https://www.sbert.net/) installed. Then, you can use the pre-trained models like this:
32
- ```python
33
- from sentence_transformers import CrossEncoder
34
- model = CrossEncoder('model_name', max_length=512)
35
- scores = model.predict([('Query', 'Paragraph1'), ('Query', 'Paragraph2') , ('Query', 'Paragraph3')])
36
- ```
37
-
38
-
39
- ## Performance
40
- In the following table, we provide various pre-trained Cross-Encoders together with their performance on the [TREC Deep Learning 2019](https://microsoft.github.io/TREC-2019-Deep-Learning/) and the [MS Marco Passage Reranking](https://github.com/microsoft/MSMARCO-Passage-Ranking/) dataset.
41
-
42
-
43
- | Model-Name | NDCG@10 (TREC DL 19) | MRR@10 (MS Marco Dev) | Docs / Sec |
44
- | ------------- |:-------------| -----| --- |
45
- | **Version 2 models** | | |
46
- | cross-encoder/ms-marco-TinyBERT-L-2-v2 | 69.84 | 32.56 | 9000
47
- | cross-encoder/ms-marco-MiniLM-L-2-v2 | 71.01 | 34.85 | 4100
48
- | cross-encoder/ms-marco-MiniLM-L-4-v2 | 73.04 | 37.70 | 2500
49
- | cross-encoder/ms-marco-MiniLM-L-6-v2 | 74.30 | 39.01 | 1800
50
- | cross-encoder/ms-marco-MiniLM-L-12-v2 | 74.31 | 39.02 | 960
51
- | **Version 1 models** | | |
52
- | cross-encoder/ms-marco-TinyBERT-L-2 | 67.43 | 30.15 | 9000
53
- | cross-encoder/ms-marco-TinyBERT-L-4 | 68.09 | 34.50 | 2900
54
- | cross-encoder/ms-marco-TinyBERT-L-6 | 69.57 | 36.13 | 680
55
- | cross-encoder/ms-marco-electra-base | 71.99 | 36.41 | 340
56
- | **Other models** | | |
57
- | nboost/pt-tinybert-msmarco | 63.63 | 28.80 | 2900
58
- | nboost/pt-bert-base-uncased-msmarco | 70.94 | 34.75 | 340
59
- | nboost/pt-bert-large-msmarco | 73.36 | 36.48 | 100
60
- | Capreolus/electra-base-msmarco | 71.23 | 36.89 | 340
61
- | amberoad/bert-multilingual-passage-reranking-msmarco | 68.40 | 35.54 | 330
62
- | sebastian-hofstaetter/distilbert-cat-margin_mse-T2-msmarco | 72.82 | 37.88 | 720
63
-
64
- Note: Runtime was computed on a V100 GPU.
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+ # fork of Cross-Encoder for MS Marco
5
+ https://huggingface.co/cross-encoder/ms-marco-MiniLM-L-12-v2
6
 
7
+ Added a custom handler to do reranking.