Spaces:
Running
on
CPU Upgrade
model resubmit needed (1 model returning as finished)
Hi! We submitted 4 models to the leaderboard this afternoon - two from Valiant Labs, two from sequelbox.
3 of the 4 are running, but 1 (ValiantLabs/ShiningValiant) is immediately returning as 'finished' and not running.
It hasn't been run, though, it's not on the leaderboard.
Can you start this run for us? Thanks!
most recent re-submission:
https://huggingface.co/datasets/open-llm-leaderboard/requests/commit/340023e9835a855737461592a2c2bc5158e6b04b
https://huggingface.co/datasets/open-llm-leaderboard/requests/commit/3f02fdf956c7f0d0bd6ba3260ff4b6c7aca82cfc
I checked your logs and request file - why is the "revision" commit 1.3?
It's not a valid commit hash, hence the failure (I'll check why it appears as finished though) - I'll also make it clearer
Thank you for providing the links to the request file!
@clefourrier I appreciate the new wording on the submit form, thank you :)
@clefourrier (CC: @SaylorTwift )
hmm, the 'FINISHED' problem is still happening for Shining Valiant.
tried submitting with blank revision field as well as the exact commit.
most recent:
https://huggingface.co/datasets/open-llm-leaderboard/requests/commit/216245b04a0b542e8ed0a695b01b63d4c249abd6
https://huggingface.co/datasets/open-llm-leaderboard/requests/commit/e286862b8e96fa54e67cad075fe75b6c323bdcd3
Your model is running fine (about 10h remaining if it doesn't crash or get cancelled :) )
Interestingly, it returned as FINISHED when it got cancelled (and before it was re-ran) not FAILED because you already submitted a model of the same name 2 months ago - and it is checking the results file against names. We'll fix this.
thanks for clarifying what was happening!
we're still waiting for results; assuming there's been some crashes? :) no worries, we know there's a lot of people excited about their models.
I'll keep an eye on the results repo and watch out there.
thanks again!