12 GB VRAM
Collection
Quants that run fast on card with 12GB of VRAM and 4096 context length
•
6 items
•
Updated
•
1
BigMaid-20B-v1.0
exllamav2 quant for TeeZee/BigMaid-20B-v1.0
Should run on 12 GB of VRAM cards in webui with context length set to 4096, ExLlamav2_HF loader and cache_8bit=True
All comments are greatly appreciated, download, test and if you appreciate my work, consider buying me my fuel:
- Downloads last month
- 15
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.