Running large language models like ChatGPT on a single GPU
> ...a high-throughput generation engine for running large language models with limited GPU memory (e.g., a 16GB T4 GPU or a 24GB RTX3090 gaming card!).laughs in 6 gb vram and no tensor cores.