You can get all gradients by iterating the parameters after the backward operation:
for name, param in model.named_parameters():
    print(name, param.grad)

Using this approach you can store the gradient magnitude and plot it for each iteration using e.g. matplotlib.
To get the learning rate and other internal states you can use optimizer.state_dict() or optimizer.param_groups[id] and check the values you would like to track.