I am confused about a couple things:
For each batch you get one loss value, so why should I average over all the samples instead of over the batches?
Wouldn’t that give me a very small loss value? Since I am adding 64 loss values (64 batches) and dividing by all the images (8000 more or less)?
Also how can I decide which values to normalize with is 0.5 a good choice by default?