Thanks I figure out what I was doing wrong. I was using test_predicted and names in the confusion matrix calculations these values keep changing at each iteration of the data loader after each batch. to fix this I created a list to store each of them and attempted
value = []
valuename = []
with torch.no_grad():
for b, (pics, names) in enumerate(test_loader):
        if torch.cuda.is_available():
            pics = pics.cuda()
            names = names.cuda()
            
        b+=1

        # Apply the model
        y_val = MobileNet(pics)
        val_loss = criterion(y_val, names)
        # Tally the number of correct predictions
        test_predicted = torch.max(y_val.data, 1)[1] 
        value.append(test_predicted)
        valuename.append(names)
        tst_corr += (test_predicted == names).sum()
        
        test_correct.append(tst_corr)


confusion_matrix(value, valuename)
Unfortunately, that gives me this:  ValueError: unknown is not supported