Noa Roggendorff

Ah we can work with that, then the issue is that the space is incomplete/misconfigured, (i would reccomend amending your original post to avoid confusion).

I just read your blog post: https://huggingface.co/blog/nroggendorff/train-with-llama-architecture

It provides some useful context, thanks.

From reading the dockerfile and image file, it appears that cuda was never included in the image.

You may find the following resources helpful for using docker with spaces:
https://huggingface.co/docs/hub/en/spaces-sdks-docker

If you are using cuda, this may also help inform on how to setup cuda, and also test if cuda works (with docker):
https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

References
https://huggingface.co/spaces/nroggendorff/train-llama/blob/main/Dockerfile

https://hub.docker.com/layers/nroggendorff/train-llama/latest/images/sha256-8cd7859f8a7cc3b669b344e87fa342e3c464e449141e267fbb52cfb48c32310f

Hope you find this helpful,
Let me know if you have any more questions, let me know here or email me.

The base image for that Dockerfile has Cuda installed and configured.

You are welcome to open a PR with your proposed fix on https://github.com/nroggendorff/train-llama.

Ah we can work with that, then the issue is that the space is incomplete/misconfigured, (i would reccomend amending your original post to avoid confusion).

I just read your blog post: https://huggingface.co/blog/nroggendorff/train-with-llama-architecture

It provides some useful context, thanks.

From reading the dockerfile and image file, it appears that cuda was never included in the image.

You may find the following resources helpful for using docker with spaces:
https://huggingface.co/docs/hub/en/spaces-sdks-docker

If you are using cuda, this may also help inform on how to setup cuda, and also test if cuda works (with docker):
https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

References
https://huggingface.co/spaces/nroggendorff/train-llama/blob/main/Dockerfile

https://hub.docker.com/layers/nroggendorff/train-llama/latest/images/sha256-8cd7859f8a7cc3b669b344e87fa342e3c464e449141e267fbb52cfb48c32310f

Hope you find this helpful,
Let me know if you have any more questions, let me know here or email me.

The base image for that Dockerfile has Cuda installed and configured.

I am not sure if that makes sense, I am under the impression that, if the space is not running(not started), no models can be actively loaded in the space.

Can you share your relevant workflow(docker-compose, app code, etc) so i can see more clearly whats happening?

I might be able to aid in a solution, its possible that there is an issue in the workflow itself.

EDIT: I looked at the spaces, Do you mean this space as an example? 'https://huggingface.co/spaces/nroggendorff/train-llama'
Because this space shows a missing "CUDA_HOME" env var, most your other spaces throwing errors about missing CUDA drivers or are paused. These are configuration errors.

Could you tell me the space and error message?
I might be able to help you fix it.

That’s the one.

what the~

it's pretty specific to my workflow, but spaces now don't get cuda until after they start, so you can't load models or anything until an app is running

403

it's a similar architecture to image generation, so.. kinda? diffusion llms aren't very popular though, so there isn't a ton of research on them. transformers is a much more reliable model type for now.

edit: it's not really a super serious experiment, they are more for testing if a logical response is possible this way.
this is also kinda why q and a bots are really bad, people just found that that format doesn't scale very well at all

edit 2: (i said one of, because another huge reason is quality data scarcity and lack of flexibility. with incremental models like gpts, you can have any number of roles and stuff, whereas input-output models just have that)