Deep Learning, Artificial Intelligence, Convolutional Network, Image Classification, Resnet.
activations and their derivatives for the top layer in the sequence. Only activations of non-reversible layers, such as subsampling layers, need to be stored explicitly during backprop, which is handful. In this case, the storage cost of the activations would be small, and independent of the depth