I would recommend to store all predictions and targets in a list (don’t forget to wrap this code in a with torch.no_grad() block or detach the predicitons) inside the loop.
Once you’ve collected all predictions and the corresponding target, you could use sklearn.metrics.confusion_matrix (or search here in the board for a PyTorch implementation we’ve posted some time ago).
To get the per-class accs, you could use:
conf = confusion_matrix(targets, preds)
conf = conf / conf.sum(axis=1)[:, np:newaxis]
print(conf.diagonal())