Hi Alban,
Here, my idea is to replace input, which supposes to consume a lot of memory (if the mini batch-size is set to be large), by several compact tensors, which will be used to estimate the input later in the backward pass. Is there any ways that kill input in the forward pass, and utilize these compact tensors to reconstruct input in the backward pass?
Thank you so much for your reply.