I’m using CNN2D for text sentiment analysis with the following settings:

optimizer = optim.Adam(model_pyt.parameters())
criterion = nn.BCEWithLogitsLoss()

I’m currently training various neural networks and I’m trying to ascertain how the gradient is behaving at each training step and thus would like to plot the gradient magnitude vs time step to observe if the gradient is vanishing or exploding.  Is there a way to do this?  Also is there a way to plot the learning rate per training step so we can observe how that’s changing, as it should get smaller as we reach a stationary point on the convex loss function?