Solved by MariosOreo in post #2 
              
              
                Hi, @likethevegetable 
I tired a snippet as follows: 
random_input = torch.randn(3,3)
random_output = torch.randn(3,3)
criterion = nn.MSELoss()
# model
l1 = nn.Linear(3, 10, bias=False)
l2 = nn.Linear(10, 3, bias=False)
intermediate = l1(random_input)
# intermediate.requires_grad=False  RuntimeError…