For 2, 3, I think this might be a good starting point, autograd mechanics: https://pytorch.org/docs/stable/notes/autograd.html 7
For 1, I do not have a good answer. Normally Adam is more forgiving towards hyperparameter choices as compared to SGD. It is useful when you want to iterate quickly on a problem (to see if it works or not)
For 4, I think you can use tensorboard https://pytorch.org/docs/stable/tensorboard.html?highlight=tensorboard 2
or this package: https://github.com/szagoruyko/pytorchviz 1