Title: Hi HN,<p>Over the last few months I&#x27;ve been working with some folks on a tool named Ollama (<a href="https:&#x2F;&#x2F;github.com&#x2F;jmorganca&#x2F;ollama">https:&#x2F;&#x2F;github.com&#x2F;jmorganca&#x2F;ollama</a>) to run open-source LLMs like Llama 2, Code Llama and Falcon locally, starting with macOS.<p>The biggest ask since then has been &quot;how can I run Ollama on Linux?&quot; with GPU support out of the box. Setting up and configuring CUDA and then compiling and running llama.cpp (which is a fantastic library and runs under the hood) can be quite painful on different combinations of linux distributions and Nvidia GPUs. The goal for Ollama&#x27;s linux version was to automate this process to make it easy to get up and running.<p>The is the first Linux release! There&#x27;s still lots to do, but I wanted to share it here for to see what everyone thinks. Thanks for anyone who has given it a try and sent feedback!
Upvote: