I try to save the memory on GPU by .cpu() using the code like the following, but failed.
import ...

model = MyModel(...)
loss = Myloss(...)

def train():
   model.to('cuda')
   loss.to('cuda')

   model.train()

   # optimize parameters

   model.cpu()
   loss.cpu()

The memory cost does not decrease. Why does .cpu() fail to move the parameters from the GPU
memory to the memory of CPU?