Your code snippet doesn’t show any part which might store the computation graph.
Maybe this line of code could be problematic (I haven’t tested it):
writer.add_scalar('val_loss', val_loss,i)

Could you use writer.add_scalar('val_loss', val_loss.item(), i) and rerun the code?
Also, did you verify that the GPU memory is increasing in each iteration via nvidia-smi?