Not Found: Model not found
Hello there, I wasn't sure if this was something that needed to be brought to your attention, but the vast majority, if not all of your models, seem to not be working as off the past hour. If one attempts to use the inference API, it produces error such as "error failed to fetch blob" or something like "this model does not exist" or "not found: model not found". In my spaces, the error logs are showing JSONDecoderErrors. I am not sure if this is something on Huggingfaces end, or if it is something you need to fix on your end (in which case if it is, I do not envy you having to update over 3,000 models.)
Thanks for the report!
I think it's probably a problem on the Hugging Face side.😅 There have been reports of abnormalities related to the Inference API in various places over the past few days.
Hello there, according to status.huggingface.co, the inference API service did experience issues recently, but this was only a few days ago, and it has since then been resolved. If you search for models under your name and filter them for text-to-image and running on the HF Inference API only, only one model is currently loaded, and using the inference text box produces "not found: model not found". Interestingly enough, if you use a space such as your Diffusecraft space which has a Zero GPU, a model of yours can be loaded and generate successfully, but the deployment status on the actual model page still state "This model is not currently available via any of the supported Inference Providers." My spaces are also producing this error for every model during the inference API call in the generation process:
Traceback (most recent call last):
File "/home/user/app/app.py", line 63, in gen_fn
result = models_loadmodel_str
File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1122, in call
outputs = client_utils.synchronize_async(
File "/usr/local/lib/python3.10/site-packages/gradio_client/utils.py", line 540, in synchronize_async
return fsspec.asyn.sync(fsspec.asyn.get_loop(), func, *args, **kwargs) # type: ignore
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 103, in sync
raise return_result
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 56, in _runner
result[0] = await coro
File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1553, in process_api
result = await self.call_function(
File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1191, in call_function
prediction = await anyio.to_thread.run_sync(
File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2461, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 962, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 659, in wrapper
response = f(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/gradio/external.py", line 415, in query_huggingface_api
errors_json = response.json()
File "/usr/local/lib/python3.10/site-packages/requests/models.py", line 978, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Hope this helps.
The “not found” is too mysterious...
Edit:
I reported it to HF Discord.
I wish I had more information, but all else that I have is a screenshot of the only model that is supposedly deployed and the error it produces:
That's a great tip, even better than you think. It's LoRA. And the base model Kivotos is broken...
https://huggingface.co/yodayo-ai/kivotos-xl-2.0?text=test
This particular model is the only model that is supposedly deployed via the HF Inference API that is one of your models. It is not one the models used in my spaces, but the issue persists not only in my spaces, but all CPU based spaces that use your models. This is a very strange issue.
Actually, there are still some models that are causing problems even with the LLM. Some of the models in the Inference API are clearly malfunctioning in ways I've never seen before.
Edit:
The HF staff are currently verifying the matter, including the LLM.
Have the staff been able to determine why the Loras are breaking? I noticed that they added another third-party API, but I wonder if they updated the API in a way that caused everything to break, such as an errant comma where a period should be, or a typo.