I don’t think labels is hot encoded on:
for images, labels in validation_dataset.unbatch(): # unbatch the test data and get images and labels
  y_labels.append(labels.numpy().argmax()) # append the index which has the largest value (labels are one-hot)

so that argmax makes every item in y_labels be a 0 (the first item is always the biggest one)
if you remove the argmax the confusion matrix will make more sense
by the way, you could also use the confusion matrix from TensorFlow if you want,  there’s an example here 2.