CP = torch.load(PATH)
net.load_state_dict(CP['model_state_dict'])
optimizer.load_state_dict(CP['optimizer_state_dict'])
net.eval()

sample = train_X[0:0 + BATCH_SIZE].view(-1, 1, 50, 50)
sample = sample.to(device)
with torch.no_grad():
    output = net(sample)
    print(output, "=?", y)


output:
tensor([[2.0367e-11, 1.0000e+00],
        [2.8479e-02, 9.7152e-01],
        [1.4075e-14, 1.0000e+00],
        [9.9997e-01, 3.3164e-05],
        [5.6208e-08, 1.0000e+00],
        [1.0000e+00, 1.1212e-07],
        [1.3695e-03, 9.9863e-01],
        [6.8686e-02, 9.3131e-01],
        [5.6444e-09, 1.0000e+00],
        [1.7498e-04, 9.9983e-01],
        [2.0622e-13, 1.0000e+00],
        [1.8598e-21, 1.0000e+00],
        [8.6885e-01, 1.3115e-01],
        [4.4393e-01, 5.5607e-01],
        [9.9974e-01, 2.5576e-04],
        [3.4825e-09, 1.0000e+00],
        [8.9833e-02, 9.1017e-01],
        [2.6258e-04, 9.9974e-01],
        [9.9997e-01, 2.8542e-05],
        [1.5574e-01, 8.4426e-01],
        [7.8556e-06, 9.9999e-01],
        [3.8838e-05, 9.9996e-01],
        [9.1441e-01, 8.5585e-02],
        [9.9998e-01, 2.0286e-05],
        [3.4435e-04, 9.9966e-01],
        [3.5586e-05, 9.9996e-01],
        [1.3090e-11, 1.0000e+00],
        [3.3178e-06, 1.0000e+00],
        [8.9947e-02, 9.1005e-01],
        [3.9299e-06, 1.0000e+00],
        [1.4769e-01, 8.5231e-01],
        [1.5370e-04, 9.9985e-01],
        [8.2074e-01, 1.7926e-01],
        [9.9904e-01, 9.6462e-04],
        [2.9493e-03, 9.9705e-01],
        [7.6622e-03, 9.9234e-01],
        [4.7667e-01, 5.2333e-01],
        [1.0248e-01, 8.9752e-01],
        [1.2424e-11, 1.0000e+00],
.....

This gives particularly different outputs, but not so success, though! When I applied on images one by one, not batches, it gives:
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[1., 0.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[0., 1.]], device='cuda:0')
tensor([[1.0000e+00, 1.7080e-21]], device='cuda:0')
...

However, this is not the effect of with torch.no_grad() . Outputs are same without it. I notice now, one or two are different in the output list and they are actually different number when testing batches, so I doubt this is because of code.