Llama2.c: Inference llama 2 in one file of pure C
This running in the browser via Emscripten by Georgi Gerganov of llama.cpp fame:<p><a href="https:&#x2F;&#x2F;ggerganov.com&#x2F;llama2.c&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;ggerganov.com&#x2F;llama2.c&#x2F;</a><p>Via his Twitter with ongoing thread: <a href="https:&#x2F;&#x2F;twitter.com&#x2F;ggerganov&#x2F;status&#x2F;1683174252990660610" rel="nofollow noreferrer">https:&#x2F;&#x2F;twitter.com&#x2F;ggerganov&#x2F;status&#x2F;1683174252990660610</a><p>This and the original is all absolutely awesome, it&#x27;s obviously only a proof of concept with a tiny model, but <i>local first</i> LLMs are really exciting. I particularly love the idea of being able to build webapps with local inference.<p>With optimisation, research into ways to make smaller models, partial downloads, and then the opportunity to use WebGPU we potentially have the start of an exciting new way to build privet local LLM based apps.<p>It&#x27;s never going to be up to the same capabilities of hosted LLMs on massive clusters of top end GPUs, but there are so many use cases that this sort of thing will enable.