I read my own question again (typed while I was having some other work; Sorry for that). My question is somewhat difficult to comprehend when seeing for the first time. So, let me ask questions one by one. First question:
How do I access all weights of a model? Currently, I am accessing with layer names like model.conv1.weight.grad. But, is there anything like model.weight.grad to access all gradients (setter and getter)?
To all PyTorch developers, thanks for the beautiful library and to those helping, thanks for your time.