As said I’m not trying to escape calculating the layers between the desire tensor and output, but on earlier layers.
One thing that I’m never sure about, does requires_grad_(False)  block the gradient from moving to earlier layers?
lets say I want the gradient just for conv3:
x = self.conv1(x)
x = self.conv2(x)
x = self.conv3(x)
out = self.conv4(x)

Should this be enough:
self.conv2.requires_grad_(False))

or this is a must:
self.conv1.requires_grad_(False))
self.conv2.requires_grad_(False))

Thanks.