Thanks @KaiHoo, just keeping correct and total labels on CPU is OK, but you do it for all batches like that. On the other hand the formula in approach 2 I saw in here 1 and I slightly modified it. It is good if you need per batch accuracy, but more important is to have per epoch accuracy. Is there any nice example for cifar10 that uses nice accuracy metrics practice, I would be happy if someone share.
For the epoch accuracy do you just average the per batch accuracy with drop_last=False or you keep the correct count?