I need to somewhat move the activations to cpu after the computation graph is generated. For instance, some computation graphs  are generated in the GPU as follows:

input1, input2 = torch.randn(128,3,32,32)
output1 = model(input)
output2 = model(input)

which would need the memory of about 2M with 2 computational graphs generated. Now I need to place the activations associated with output1 to cpu, such that the memory in GPU would fall down to about M. This should make sense, but I don’t know how to implement it. Thanks!