In the usual use case you would compute the average of both, the training and test loss.
Your suggested approach of using:
    loss = criterion(outputs_ts, labels_ts)
    losses_sum += loss.item()
    num_samples += 1
...
loss_test_avg = losses_sum / num_samples

would work for both losses. Although you might see a slight bias (as explained before), it could be “good enough”.
Alternatively, you could use the AverageMeter 2 from the ImageNet example and update the loss as seen here 3.