There is a flaw in my earlier explanation
Running loss should be
running_loss += loss.item() * inputs.size(0)
Not
running_loss += loss.item()
Contrary to what I earlier said, the loss returned is an average over all examples in a batch, given this, you could simply divide the running loss by the number of batches, however since the final batch is often not the same size with others, your results would be inaccurate, hence, the solution is to reverse the average by multiplying the loss in a batch by the total number of images in the batch, this gives unaveraged sum of losses over all examples, when you divide the running loss by the total number of images ,you get accurate loss metrics.
The line
dataset_sizes[phase]
Uses the approach suggested by @roaffix
It returns the number of Images in the dataset.