Isn’t this the expected behavior when setting shuffle=False?
shuffle=False means, now the data is no longer shuffled but in-order.
The labels before tensor you printed above is probably only your first batch and thus only contains label 1. If you were to print further batches they would probably contain your other labels.
I am guessing your test_set is a torchvision.datasets.ImageFolder() with your images being inside different folders representing your labels?
If so than this is indeed the expected behavior.