Parameter manipulation can be performed inside a with torch.no_grad() block.
I’m not sure to understand the requirement about deterministic results properly.
Do you want to make a GPU run match a CPU run or just GPU to GPU and CPU to CPU?