Hi,
Is there a way to access the update values computed by the optimizer?
For example, if I’m using plain SGD with a learning rate a, then update values are simply a*gradients.
More concretely, these values should be what .backward() method computes, for each instance in the batch, and automatically adds to the model weights.