Actually, this part of the code looks suspicious:
        # Get inputs
        inputs, targets = data

        inputs = inputs.to(device)
        targets = targets.to(device)


        sample_fname = testloader.dataset.samples[i][0]

If the dataloader shuffles while the underlying dataset is not shuffled, then this explains why the dataset shows the first four filenames each time while the data is correctly shuffled.
If you need to use the filenames along with the image data, you might consider modifying the DatasetFolder class to return the filename along with the raw inputs and targets.
https://pytorch.org/vision/stable/_modules/torchvision/datasets/folder.html#DatasetFolder 4