Artificial neural networks (ANNs) lack in biological plausibility, chiefly because backpropagation requires a variant of plasticity (precise changes of the synaptic weights informed by neural events that occur downstream in the neural circuit) that is profoundly incompatible with the current understanding of the animal brain. Here we propose that backpropagation can happen in evolutionary time, instead of lifetime, in what we call neural net evolution (NNE). In NNE the weights of the links of the neural net are sparse linear functions of the animal's genes, where each gene has two alleles, 0 and 1. In each generation, a population is generated at random based on current allele frequencies, and it is tested in the learning task. The relative performance of the two alleles of each gene over the whole population is determined, and the allele frequencies are updated via the standard population genetics equations for the weak selection regime. We prove that, under assumptions, NNE succeeds in learning simple labeling functions with high probability, and with polynomially many generations and individuals per generation. We test the NNE concept, with only one hidden layer, on MNIST with encouraging results. Finally, we explore a further version of biologically plausible ANNs inspired by the recent discovery in animals of dopaminergic plasticity: the increase of the strength of a synapse that fired if dopamine was released soon after the firing.
First of all, I must confess that my knowledge is quite limited to read this paper. Perhap the authors present something that I can not catch up at the present.
I conjecture the paper would like to bring the evolution in genetics and perhap brain cirecuits as well to define a novel neural net model, called NNE by the authors. 
The paper is somewhat cumbersome in the introduction that makes the reader (here is myself) can not understand the main idea. At first the authors introduce about evolution in genetics and genomics that is a bit different from what I known. Then the authors claim that they can show that the brain circuits can evolved in their model.
There are so many mistake and/or typos in sentences and in mathematical formulation. These make me can not finish reading the paper. I have to stop reading at the end of Section 2.
Here are my concerns and questions:
1). What is "STDP" in the 2nd papragraph in Introduction?
2). in the 3rd papragraph in Introduction: "Suppose that the brain circuitry for a particular classiﬁcation task, such as “food/not food”,is encoded in the animal’s genes, assuming each gene to have two alleles 0 and 1". This is realistics, the allels in animal is 0 1 or 2 if encoded.
3). The authors use the words "gene, genotype, phenotype" in a special way that is different to what I known in genomics (in GWAS).
4).  in the 3rd papragraph in Introduction: "At each generation, a gene is an independent binary variable with ﬁxed probability of 1". What do you mean by fixed probability of 1? I can not understand in anysense that I know.
5). In Section 2, What is n? the authors start the mathematical formulation but I can not find out what is n? Is it the sample size?
6). In Section 2, paragraph 2, you define y~ \mathcal{D}, BUT then in all formulas later you denote y ~ D. What is D??? I can not understand.
7).  In Section 2, paragraph 2, you define a label of y as \ell(y), BUT then in the 1st sentence of the 3rd paragraph in Section 2 you wrote    L(NNE_x(t), l(y)). What is l(.) here ???
8). In the equations (1) and (2), what is p^t  ???  You have NOT defined it.
9). Right after equations (1) and (2), What is \epsilon ???? Can NOt understand.
10). The sentence right after the equation (3): "This is the standard update rule in population genetics under the weak selection assumption." This is NOT trivial to me, and even the machine learning comunity, we do not know this rule, it is not obvious. PLEASE provide exact reference.
11). the first equation in the PROOF of LEMMA 1 wrong  \mathcal{L} (p^t)  should be equal to E_{x~p^t}  NOT p.
12). in the PROOF of Theorem 1, I can NOT find out where \gamma has been defined?
13). What is d  in Theorem 2? is it the dimension? I make too many guesses !!!