Are you shuffling the DataLoader?
If the length of your dataset is not divisible by the batch size without a remainder, you might see small differences in your validation accuracy. The last batch might be smaller, which creates a bias using your normalization (dividing by len(loader)).
Try this code instead:
    acc = torch.eq(preds, labels).sum().item()
    avg_acc += acc
    avg_loss += loss.item() * images.size(0)

avg_acc /= len(loader.dataset)
avg_loss /= len(loader.dataset)