  In this brief, we show that sequentially learning new information presented
to a continual (incremental) learning model introduces new security risks: an
intelligent adversary can introduce small amount of misinformation to the model
during training to cause deliberate forgetting of a specific task or class at
test time, thus creating "false memory" about that task. We demonstrate such an
adversary's ability to assume control of the model by injecting "backdoor"
attack samples to commonly used generative replay and regularization based
continual learning approaches using continual learning benchmark variants of
MNIST, as well as the more challenging SVHN and CIFAR 10 datasets. Perhaps most
damaging, we show this vulnerability to be very acute and exceptionally
effective: the backdoor pattern in our attack model can be imperceptible to
human eye, can be provided at any point in time, can be added into the training
data of even a single possibly unrelated task and can be achieved with as few
as just 1\% of total training dataset of a single task.

Machine Learning;Cryptography and Security