Exploring Weight Agnostic Neural Networks
how is this different from boring old evolutionary algorithms?<p>In my opinion the big breakthrough that enabled optimization and machine learning was the discovery of reverse mode automatic differentiation, since the space or family of all possible decision-functions is high dimensional, while the goal (survival, reproduction) is low dimensional. Unless I see a mathematical proof that evolutionary algorithms are as efficient as RM AD, I see little future in it, and apparently neither did biology since it decided to create brains.<p>It&#x27;s not an ideological stance I take here (of nature vs nurture).<p>For simplicity, lets pretend humans are single-cellular organisms, what does natural selection exert pressure on? our DNA code: both the actual protein codes and the promotor regions. I claim that variation on the proteins are risky (a modification in a proteinn coding region could render a protein useless) while a variation on the promotor regions is much less risky: altering a nucleotide there would slightly affect the affinity modulating transcription, so the cell would behave essentially the same but with different treshold concentrations, think of continuous parameters that describe our body (assuming same nurture, food, etc) some people are a bit taller, some people a bit stronger, etc... so how many of these continuous parameters do we have? On the order of the same number as the total number of promotor regions in DNA in the fertilized egg: both on human DNA and in one mitochondria (assuming there isn&#x27;t a chemical signals addressing and reading and writing scheme for say 10 mitochondria)...<p>EDIT: just adding that for a certain fixed environment, there are local (and a global) optimum of affinity values for each protein, so that near a local optimum the fitness is roughly shaped like -s(a-a_opt)^2 where s is spread and a_opt the local optimum affinity value. In other words, it is not so that &quot;better affinity&quot;, means fitter, not at all, a collection of genomes from an identical environment will hover around an affinity sweet spot.<p>According to wikipedia [0] that would result in about<p>about 2x 20412 &quot;floats&quot; for just protein-coding genes<p>about 2x 34000 &quot;floats&quot; when also including the pseudo-genes<p>about 2x 62000 &quot;floats&quot; when also including long ncRNA, small ncRNA, miRNA, rRNA, snRNA, snoRNA<p>these &quot;floats&quot; are the variables that allow a species to modulate the reaction constants in the gene regulatory network, since natural selection can not directly modulate the laws of physics and chemistry, and modulating the protein directly instead of the promotor region affinities &#x2F; reaction rates risks disfunctional proteins...<p>so my estimate of <i>an</i> upper limit of the number of &quot;floats&quot; in the genetic algorithm is ~120000 (and probably much less if not each of the above has a promotor region).<p>thats not a lot of information, if we think about the number of synaptic weights in the brain, and many of these are shared <i>in utilization</i> by the other cell types besides neurons.<p>I consider the possibility that: sperm cell, egg cell, or fertilized egg cell performs a kind of POST (power-on-self-test) that checks for some of the genes, although simply reaching the fertilized state may be enough of a selftest so no spontaneous abortion test may be needed (to save time and avoid resources spent on a probably malformed child).<p>[0] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Human_genome#Molecular_organization_and_gene_content" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Human_genome#Molecular_organiz...</a><p>EDIT2: regarding:<p>&gt;This makes WANNs particularly well positioned to exploit the Baldwin effect, the evolutionary pressure that rewards individuals predisposed to learn useful behaviors, without being trapped in the computationally expensive trap of ‘learning to learn’.<p>The computationally expensive trap of having to &#x27;learn to learn&#x27; could end up being as mundane as a low number of hormones to which neurons in the brain globally or collectively respond, which enables learning by reward or punishment, and from then on anticipating reward or punishment, and our individual end goal stems from this anticipation, and anticipating the anticipation etc...