Hi @Yaroslav_Bulatov,
Setting requires_grad=False should work in your use case. It won’t stop autograd to run the backprop on the rest of the model, and only compute intermediate buffers needed by the rest of the computations.