Machine Learning, Software Development, Artificial Intelligence, Technology, Data Science.
offers a better space vs perplexity tradeoff than 3b (at least with simple post-training quantization). I haven’t dug into the price-performance of CPU vs GPU inference recently[1], but this certainly seems useful for the CPU case. I’m still pretty certain that we’ll eventually be using vector