Yes, sure if you want to update weights using gradients from the backward pass of loss1 you should put optimizer.step() after loss1.backward()
But again, I’m not sure about influence on speed of this approach, you can try to measure it by comparing these snippets
import time
start_time = time.time()
loss = loss1 + loss2

optimizer.zero_grad()
loss.backward() 
optimizer.step()

end_time = time.time() - start_time
print("%s seconds" % (end_time))

and this one
import time
start_time = time.time()

optimizer.zero_grad()
loss1.backward() 
optimizer.step()

optimizer.zero_grad()
loss2.backward()
optimizer.step()

end_time = time.time() - start_time
print("%s seconds" % (end_time))