ptrblck:

pass will store the intermediate activations, which would be needed to compute the gradients in the backward pass. The backward pass will then accumulate the gradients to each used parameter.


I see, thank you!
So will the intermediate activations be accumulated or replaced?
In other words, if I did:
model.forward(validation_data)
model.forward(train_data)
And then I proceeded to do a backwards pass and update weights, would the activations from the second forward pass only be used to compute gradients?
Thanks!