You should create a confusion matrix for each dataset, i.e. create the training confusion matrix right after the for epoch loop and print it after the DataLoader loop, then create a validation confusion matrix after model.eval() and print it after the validation DataLoader loop.