Indeed, += is an in-place operation, you can’t use it with variables requiring gradients:
v1 = Variable(torch.rand(5,5), requires_grad=True)
v2 = Variable(torch.rand(5,5), requires_grad=True)
v1 += v2

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: a leaf Variable that requires grad has been used in an in-place operation.