Llama 2 on ONNX runs locally
For anyone running locally, could you please describe your hardware setup?  CPU only? CPU+GPU(s)?  How much memory? What type of CPU? Particularly interested in larger models (say >30b params).<p>For transparency, I work for an x86 motherboard manufacturer and the LLM-on-local-hw space is very interesting.  If you're having trouble finding the right HW, would love to hear those pain points.