Spaces:
Running
on
CPU Upgrade
MultiLongDocRetrieval dataset failed to load
I would like to ask if there have been any situations where the MultiLongDocRetrieval dataset failed to load, and how to solve this problem: ConnectionError: Couldn't reach https://huggingface.co/datasets/Shitao/MLDR/resolve/main/mldr-v1.0-zh/corpus.jsonl.gz (ConnectTimeout (MaxRetryError ("HTTPSConnectionPool (host='huggingface. co ', port=443)): Max retries exceeded with URL:/datasets/Shipping/MLDR/resolve/main/mldr v1.0 zh copus. jsonl. gz) Caused by ConnectTimeoutError (<urllib3. connection. HTTPSConnection object at 0x7f4b4dc81b80>,' Connection to huggingface. co timed out.=100)
But other datasets can load normally
Hello, can you please provide the code you used to run the task ? Maybe check your network connection :)
The evaluation for English (en) runs normally for me with this code:
model_name = "average_word_embeddings_komninos"
model = SentenceTransformer(model_name)
evaluation = MTEB(tasks=["MultiLongDocRetrieval"], task_langs=['en'])
results = evaluation.run(model, eval_splits=['dev', 'test'], output_folder=f"results/{model_name}")
What I need to evaluate is the dataset of ['zh '], but only MultiLongDocRetrieval has an error. This is my code, ZNVEmbedingModel in my modelcard
model_path = "sentosa/ZNV-Embedding"
model = ZNVEmbeddingModel(model_path)
with torch.no_grad():
with autocast(dtype=torch.bfloat16):
evaluation = MTEB(task_langs=['zh'], tasks=["MultiLongDocRetrieval"])
evaluation.run(model, eval_splits=['dev', 'test'], verbosity=2, output_folder="/")
Oh, the ones above are not easy to read
model_path = "sentosa/ZNV-Embedding"
model = ZNVEmbeddingModel(model_path)
with torch.no_grad():
with autocast(dtype=torch.bfloat16):
evaluation = MTEB(task_langs=['zh'], tasks=["MultiLongDocRetrieval"])
evaluation.run(model, eval_splits=['dev', 'test'], verbosity=2, output_folder="/")
another question: My results only have one MultiLongDocRetrieval that was not evaluated successfully, but the overall ranking shows no classification results. Why is this
another question: My results only have one MultiLongDocRetrieval that was not evaluated successfully, but the overall ranking shows no classification results. Why is this
Some classification results are missing: MassiveIntentClassification and MassiveScenarioClassification.
I have identified the reasons (lack of evaluation set for 'zh CN' and network issues), and I have added my modelcard again. Thank you very much for your help