That’s exactly what I did. So you’re saying that even though I gave the optimizer only the params of the first layer and set param.requires_grad = False, the loss backpropagate all the way to the first layer?