I faced the same issue even with the latest pytorch1.7.0.a0* : “Warning: Error detected in TanhBackward.”, “RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation”. Resolved it by cloning the tensor (as you suggested).