detach() cuts away backprop graph for previous network (this is needed when net_input.grad_fn is not None, and is not related to optimizer)
clone() duplicates storage, avoiding potential problems with inplace operations.
It is possible that these were added just to be safe.