Web LLM – WebGPU Powered Inference of Large Language Models
Do LLMs have a way around the high end GPU requirements, or can CPU code potentially be much more optimized somehow?This is the only thing I can think of, not everyone will have the latest high end GPUs to run such software..