Missing config.json causing issue while trying to inference this modal using infinity
[nomic-ai/colnomic-embed-multimodal-7b](INFO: Started server process [1]
INFO: Waiting for application startup.
INFO 2025-05-20 18:28:25,010 infinity_emb INFO: infinity_server.py:84
Creating 1engines:
engines=['nomic-ai/colnomic-embed-multimodal-7b']
INFO 2025-05-20 18:28:25,013 infinity_emb INFO: Anonymized telemetry.py:30
telemetry can be disabled via environment variable
DO_NOT_TRACK=1
.
INFO 2025-05-20 18:28:25,019 infinity_emb INFO: select_model.py:64
model=nomic-ai/colnomic-embed-multimodal-7b
selected, using engine=torch
and device=None
ERROR: Traceback (most recent call last):
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 406, in hf_raise_for_status
response.raise_for_status()
File "/app/.venv/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/nomic-ai/colnomic-embed-multimodal-7b/resolve/main/config.json
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/.venv/lib/python3.10/site-packages/starlette/routing.py", line 693, in lifespan
async with self.lifespan_context(app) as maybe_state:
File "/usr/lib/python3.10/contextlib.py", line 199, in aenter
return await anext(self.gen)
File "/app/infinity_emb/infinity_server.py", line 88, in lifespan
app.engine_array = AsyncEngineArray.from_args(engine_args_list) # type: ignore
File "/app/infinity_emb/engine.py", line 306, in from_args
return cls(engines=tuple(engines))
File "/app/infinity_emb/engine.py", line 71, in from_args
engine = cls(**engine_args.to_dict(), _show_deprecation_warning=False)
File "/app/infinity_emb/engine.py", line 56, in init
self._model_replicas, self._min_inference_t, self._max_inference_t = select_model(
File "/app/infinity_emb/inference/select_model.py", line 71, in select_model
unloaded_engine = get_engine_type_from_config(engine_args)
File "/app/infinity_emb/inference/select_model.py", line 36, in get_engine_type_from_config
config_path = hf_hub_download(
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 862, in hf_hub_download
return _hf_hub_download_to_cache_dir(
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 925, in _hf_hub_download_to_cache_dir
(url_to_download, etag, commit_hash, expected_size, head_call_error) = _get_metadata_or_catch_error(
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1376, in _get_metadata_or_catch_error
metadata = get_hf_file_metadata(
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1296, in get_hf_file_metadata
r = _request_wrapper(
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 277, in _request_wrapper
response = _request_wrapper(
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 301, in _request_wrapper
hf_raise_for_status(response)
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 417, in hf_raise_for_status
raise _format(EntryNotFoundError, message, response) from e
huggingface_hub.errors.EntryNotFoundError: 404 Client Error. (Request ID: Root=1-682cc9c9-3ab935887bf5e52b5a91b47b;7b8027d0-5750-4516-af73-32cea0296e33)
Entry Not Found for url: https://huggingface.co/nomic-ai/colnomic-embed-multimodal-7b/resolve/main/config.json.
ERROR: Application startup failed. Exiting.)