I feel like the network needs random data. Perhaps random recombinations of a larger training set. It would lower the accuracy of the network, but be more resilient against these sorts of attacks. Sort of like sort algorithms that include randomness.