Try to disable shuffling in the DataLoader and compare the outputs of both code snippets.
Also make sure that the indexing in confusion_matrix is right and compare the number of class predictions to the correct value in the first code.