Matrix multiplication using only addition
This is a fascinating idea. Any real acedemic critique is over my head (and I hope others chime in), but some random thoughts:<p>- &quot;logarithm LUT then add&quot; seems delightfully simple, especially at low precision. I am going to have to read that paper too...<p>- The concerns about GPU style parallelism <i>may</i> not be as bad in &quot;alternative&quot; architectures. For instance, Centaur came up with a single, serial, but hilariously wide 32,768-bit SIMD core for inference: <a href="https:&#x2F;&#x2F;fuse.wikichip.org&#x2F;news&#x2F;3256&#x2F;centaur-new-x86-server-processor-packs-an-ai-punch&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;fuse.wikichip.org&#x2F;news&#x2F;3256&#x2F;centaur-new-x86-server-p...</a><p>- The silicon simplification also seems relevant to Samsung&#x27;s in memory computing effort: <a href="https:&#x2F;&#x2F;www.servethehome.com&#x2F;samsung-hbm2-pim-and-aquabolt-xl-at-hot-chips-33&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.servethehome.com&#x2F;samsung-hbm2-pim-and-aquabolt-x...</a><p>- I wonder if this would be relevant to llama.cpp&#x27;s CPU inference?