One problem with training a neural network end-to-end this way is that the system is susceptible to unpredictable glitches: The same principle that lets people trick a NN into [thinking a panda is a vulture](<a href="https:&#x2F;&#x2F;codewords.recurse.com&#x2F;issues&#x2F;five&#x2F;why-do-neural-networks-think-a-panda-is-a-vulture" rel="nofollow">https:&#x2F;&#x2F;codewords.recurse.com&#x2F;issues&#x2F;five&#x2F;why-do-neural-netw...</a>) can happen randomly just by differing lighting&#x2F;shadow conditions, sun glare, or who knows.<p>One can always train the network with more and more scenarios, but how do you know when to stop? How good is good enough in this regard?