Evaluation reproduction
#1
by
NohTow
- opened
Hey,
PyLate is getting merged in MTEB soon, facilitating evaluation of PyLate models directly using MTEB.
In the mean time, people might want to reproduce the evaluation results, so as for GTE-ModernColBERT, I am sharing a boilerplate to reproduce the results reported in the model card.
The boilerplate can be found here.
Please note that the reported results are with a query length of 256 except for the Pony split, where we used a query length of 32 because bigger query length yields bad results (I am not sure why, this split is a bit odd).