Did you store y_pred or any other tensor attached to the computation graph somehow?
Usually you see the memory growing if you store a tensor with the computation graph, e.g.
output = model(data)
loss = criterion(output, target)
losses.append(loss)   # will store the whole computation graph -> increased memory usage