Do d_error and g_error require gradients?
Check if with print(d_error.requires_grad).
If so, is your memory continuously increasing? Try to use X_error.detatch() or X_error.data to log the information. Maybe the logger holds some reference to the Tensors.