Hi Chris!



 chrisuzokwe:

My cls_loc layer in my pooling layer has no grad, despite its requires_grad = True.


I can’t figure out what you are doing here.  But here are two comments
that may or may not be relevant:
First, a “layer” (in your case a Linear) doesn’t have a requires_grad
property; its Parameters do (such as Linear.weight).
Second, a tensor (or Parameter) that starts out with
requires_grad = True won’t have a non-None grad property
until it’s been backpropagated through.
Consider:
>>> import torch
>>> torch.__version__
'1.7.1'
>>> _ = torch.manual_seed (2021)
>>> lin = torch.nn.Linear (4, 2)
>>> lin.requires_grad
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 778, in __getattr__
    raise ModuleAttributeError("'{}' object has no attribute '{}'".format(
torch.nn.modules.module.ModuleAttributeError: 'Linear' object has no attribute 'requires_grad'
>>> lin.weight.requires_grad
True
>>> lin.weight.grad
>>> lin (torch.randn (4)).sum().backward()
>>> lin.weight.grad
tensor([[-1.2991,  0.1929,  0.4697, -0.0257],
        [-1.2991,  0.1929,  0.4697, -0.0257]])

Best.
K. Frank