discord
John did you get my completed sONFBbuild do you got discord id love to work on it with you
My local environment is too weak to try it, but it looks interesting.
its made for 8 GB I redid all the float math for ppl like you john! and I did a good job!
I tried coding below 8gb not possible INT8 was a mess
just know I was thinking of user like you john when I build it if you getter a better card or a dgx/agx LMK its edge AI lean computing
ill respin up the math if you need me too k
I refuse to believe FP16 Q4 wont run on your system try this lean version
https://huggingface.co/ghostai1/GHOSTSONAFB/blob/main/4bitapp.py
Thank you. Well, it has a GPU with 8GB of VRAM, but since it's running Windows and I have browsers open all the time, I can only use about 6GB of VRAM...😅
I do most of my daily GPU work on Hugging Face. Zero GPU is convenient. It does have the weakness of being specific to Gradio, though...
It does have the weakness of being specific to Gradio, though...
gonna do streamlit etc
I am learning Float B with music still
also its back end it has a front end API built too ill have to find it that has a full API built into it
and fully ready for cloudflare/ngrok tunneling for CORS
hmu john ill give you access t my edge lab if you need resources for builds
It seems to work in a Zero GPU environment with a few modifications for now. Goodnight.
https://huggingface.co/spaces/John6666/GHOSTSONAFB-test
sweet
fork it send me the link
damn what version is that the math I did on this was crazy
4bit runs on anything lol I might do a tpu simulation then