New neural network architecture inspired by neural system of a worm
It makes a good headline, but reading over the paper (<a href="https:&#x2F;&#x2F;www.nature.com&#x2F;articles&#x2F;s42256-022-00556-7.pdf" rel="nofollow">https:&#x2F;&#x2F;www.nature.com&#x2F;articles&#x2F;s42256-022-00556-7.pdf</a>) it doesn’t seem biologically-inspired. It seems like they found a way to solve nonlinear equations in constant time via an approximation, then turned that into a neural net.<p>More generally, I’m skeptical that biological systems will ever serve as a basis for ML nets in practice. But saying that out loud feels like daring history to make a fool of me.<p>My view is that biology just happened to evolve how it did, so there’s no point in copying it; it worked because it worked. If we have to train networks from scratch, then we have to find our own solutions, which will necessarily be different than nature’s. I find analogies useful; dividing a model into short term memory vs long term memory, for example. But it’s best not to take it too seriously, like we’re somehow cloning a brain.<p>Not to mention that ML nets <i>still</i> don’t control their own loss functions, so we’re a poor shadow of nature. ML circa 2023 is still in the intelligent design phase, since we have to very intelligently design our networks. I await the day that ML networks can say “Ok, add more parameters here” or “Use this activation instead” (or learn an activation altogether — why isn’t that a thing?).