Hi guys, I want to put some structure on the weight matrix so I have something like this in the forward function.
class myNN(nn.Module):
    def __init__(self):
        super(myNN, self).__init__()
        self.weight = nn.Linear(a, b)
    def transform(self):
        transformed_weight = do_something(self.weight)
        return transformed_weight
    def forward(self, x):
        transformed_weight = self.transform()
        x = do_something2(transformed_weight, x)
        return x

do_something() and do_something2() create some intermediate tensors (also transformed_weight in forward() is also intermediate), will they be freed after forward() returns? I keep getting CUDA out of memory error after several iterations. Thanks.