From fmassa’s answer somewhere else (source: How does one make sure that the parameters are update manually in pytorch using modules? 10):

Whenever we have an underscore in the end of a function in pytorch, that means that the function is in-place.

If you you don’t know what in place means see albanD’s answer (source: What is `in-place operation`? 71):

An in-place operation is an operation that changes directly the content of a given Tensor without making a copy. Inplace operations in pytorch are always postfixed with a  _ , like  .add_()  or  .scatter_() . Python operations like  +=  or  *=  are also inplace operations.

as a side comment, these sort of operations don’t get added to the computation graph, so they should be used with caution.