Machine Learning, Software Development, Artificial Intelligence, Technology, Data Science.
Models They take advantage of the weight matrices being fixed to perform various optimizations for CPU inference. In particular, they store weight tiles in Z-curve order and autotune some block/tile sizes. They also support quantizing with different group sizes, storing the scales and (quantized)