Hi,
So I am using 10-fold cross-validation and the model is updated without problems, but after evaluating the first fold the parameter values won’t update. The module I am using is defined below:
class CRFlayer(nn.Module, ABC):
    def __init__(self, nodes=25):
        super(CRFlayer, self).__init__()
        self.nodes = nodes
        # Sets learnable weights
        self.Wh = nn.Parameter(Tensor([0]))
        self.Wh.requires_grad = True
        self.Wv = nn.Parameter(Tensor([0]))
        self.Wv.requires_grad = True
        self.Wd1 = nn.Parameter(Tensor([0]))
        self.Wd1.requires_grad = True
        self.Wd2 = nn.Parameter(Tensor([0]))
        self.Wd2.requires_grad = True

During the first training I can manually modify the values of the parameters without problem and they will update using something like:
with torch.no_grad():  
       self.Wh.copy_(X1)
       self.Wv.copy_(X2)
       self.Wd1.copy_(X3)
       self.Wd2.copy_(X4)

However, if I use the same code, the weights won’t update during the second iteration of cross-validation (note that as soon as I reach the last epoch of the first iteration, I don’t use the model anymore until the next iteration). I also tried to reset the parameters using this:
    elif isinstance(m, CRFlayer):
        torch.nn.init.constant_(m.Wh, 0)
        torch.nn.init.constant_(m.Wv, 0)
        torch.nn.init.constant_(m.Wd1, 0)
        torch.nn.init.constant_(m.Wd2, 0)

In both cases, the values are 0 but they don’t change later. I verified the weights using the debugger and the requires_grad field is True. Do you have any suggestions? Thanks.