You can freeze the model in multiple ways:

set the requires_grad field to False
run in no_grad mode because you don’t need gradients to flow through
just never call optimizer.step()
etc

The ones that rely on the parameters not to change might not work if you do them before sending the model to cuda. The ones that do not touch the parameters (2 and 3 above), will work either way