open-llm-leaderboard/open_llm_leaderboard · Unable to submit model, due to "Unknown model size" (vilm/Quyen-Pro-Max-v0.1)

Sep 15

vilm/Quyen-Pro-Max-v0.1, both main and revision 1366c9259b0969281125637149ed9f66cf7ce602 (safetensors) have this problem,
with identical error messages, I haven't seen this problem elsewhere for other models recently

relevant code, perhaps:
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard/blob/main/src/submission/check_validity.py#L110

it's a bfloat16, chat (dpo) model

alozowski

Open LLM Leaderboard org Sep 17

Hi @CombinHorizon ,

I'm afraid we can's submit this model since it's not in a safetensors format:

Failed to get safetensors metadata for model vilm/Quyen-Pro-Max-v0.1: 'vilm/Quyen-Pro-Max-v0.1' is not a safetensors repo. Couldn't find 'model.safetensors.index.json' or 'model.safetensors' files.

According to our documentation, you can submit only a safetensors model:
https://huggingface.co/docs/leaderboards/open_llm_leaderboard/submitting

Let me close this issue, please, feel free to open a new discussion in case of any other questions!

alozowski changed discussion status to closed Sep 17