2.  Excessive dependency on exact details of training regime: whereas all normal 

human learners acquire their native language and an understanding of the world, 
despite highly varied circumstances, neural networks often tend to be quite 
sensitive to exact details such as the order in which training items are presented 
(hence a literature on "curricula" for neural networks). Likewise, it has been known 
for three decades that they are vulnerable to a problem known as catastrophic 
interference in which earlier associations are overwritten by later associations 
(McCloskey & Cohen, 1989), making them highly sensitive to the sequence in which 
items are presented. Potential solutions continue to be proposed regularly 
(McClelland, 2019) but the issue remains. Likewise, as one recent paper (Hill et al., 
2019) put it, “the degree of generalisation that networks exhibit can depend 
critically on particulars of the environment in which a given task is instantiated”  

§
The Next Decade in AI-