 [CodeSearchNet Challenge Evaluating the State of Semantic Code Search](https://arxiv.org/pdf/1909.09436.pdf)  We provide results on the CodeSearchNet benchmark, measured in Mean Reciprocal Rank (MRR), following the code search procedure outlined in Section 3.3 of Neelakantan et al.'s ["Text and Code Embeddings by Contrastive Pre-Training"](https://arxiv.org/pdf/2201.10005.pdf).  `Candidate Size = 1,000`  | Model           | Avg   | Python | Go    | Ruby  | PHP   | Java  | JS    | |-----------------|-------|--------|-------|-------|-------|-------|-------| | Carptriever-1   | 60.24 |  65.85 | 63.29 |  62.1 |  59.1 | 55.52 | 55.55 | | Contriever      | 49.39 |  54.81 |  58.9 | 55.19 | 38.46 | 44.89 | 44.09 |   `Candidate Size = 10,000`  | Model.          | Avg   | Python | Go    | Ruby  | PHP   | Java  | JS    | |-----------------|-------|--------|-------|-------|-------|-------|-------| | Carptriever-1   | 48.59 |  55.98 | 43.18 | 56.06 | 45.62 | 46.04 | 44.66 | | Contriever      |    37 |  45.43 | 36.08 | 48.07 | 25.59 | 32.89 | 31.44 |   