Hi ptrblck@ptrblck, great thanks for your reply. I have tried to insert the code in my script, and got the runtime error:
criterion = nn.NLLLoss()
optimizer = optim.Adam(net.parameters(), lr=1e-10)

epochs = 100
for epoch in range(epochs):
    running_loss = 0.0
    for i, data in enumerate(data_loader, 0):
        input, label = data
        if torch.isnan(input) or torch.isinf(input):
            print('invalid input detected at iteration ', i)
            break
        input, label = input.unsqueeze(1).float().to(device), label.to(device)

        optimizer.zero_grad()
        outputs = net(input)
        outputs = outputs.add(epsilon)
        loss = criterion(outputs, label)

        print(loss.item())
        loss.backward()
        torch.nn.utils.clip_grad_norm_(net.parameters(), 0.5)
        print(loss.item())
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 100 == 0:
            print('[%d, %5d] loss: %.3f' %
                  (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
 in 
     18         # get the inputs; data is a list of [inputs, labels]
     19         softmax_pdf, label, index = data
---> 20         if torch.isnan(input) or torch.isinf(input):
     21             print('invalid input detected at iteration ', i)
     22             break

RuntimeError: bool value of Tensor with more than one value is ambiguous

I assume you might mean this instead:
if torch.isnan(sum(sum(input))) or torch.isinf(sum(sum(input))):
    print('invalid input detected at iteration ', i)

So I replaced the code with this one, no runtime error, but it seems like it is not the input’s problem, as I still got the same nan after a while without getting any notice of 'invalid input detected at iteration '
criterion = nn.NLLLoss()
optimizer = optim.Adam(net.parameters(), lr=1e-10)

epochs = 100
for epoch in range(epochs):
    running_loss = 0.0
    for i, data in enumerate(data_loader, 0):
        input, label = data
        if torch.isnan(sum(sum(input))) or torch.isinf(sum(sum(input))):
            print('invalid input detected at iteration ', i)
            break
        input, label = input.unsqueeze(1).float().to(device), label.to(device)

        optimizer.zero_grad()
        outputs = net(input)
        outputs = outputs.add(epsilon)
        loss = criterion(outputs, label)

        print(loss.item())
        loss.backward()
        torch.nn.utils.clip_grad_norm_(net.parameters(), 0.5)
        print(loss.item())
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 100 == 0:
            print('[%d, %5d] loss: %.3f' %
                  (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

This is what I got still:
1.4137518405914307
1.4137518405914307
[1,     1] loss: 0.001
1.4274098873138428
1.4274098873138428
1.42104172706604
1.42104172706604
1.4518089294433594
1.4518089294433594
1.4289991855621338
1.4289991855621338
1.4246625900268555
1.4246625900268555
1.3915072679519653
1.3915072679519653
1.4070930480957031
1.4070930480957031
1.490980863571167
1.490980863571167
1.5186127424240112
1.5186127424240112
1.3903168439865112
1.3903168439865112
1.455558180809021
1.455558180809021
1.4122167825698853
1.4122167825698853
1.4350025653839111
1.4350025653839111
1.4551273584365845
1.4551273584365845
1.4294865131378174
1.4294865131378174
1.4452277421951294
1.4452277421951294
1.3719713687896729
1.3719713687896729
1.4808259010314941
1.4808259010314941
1.4451615810394287
1.4451615810394287
1.4479520320892334
1.4479520320892334
1.504454255104065
1.504454255104065
1.493384838104248
1.493384838104248
1.5036394596099854
1.5036394596099854
1.4297422170639038
1.4297422170639038
1.4111098051071167
1.4111098051071167
1.3866780996322632
1.3866780996322632
1.3785628080368042
1.3785628080368042
1.379706859588623
1.379706859588623
1.4556639194488525
1.4556639194488525
1.377264142036438
1.377264142036438
1.456748127937317
1.456748127937317
1.3824539184570312
1.3824539184570312
1.4618127346038818
1.4618127346038818
1.3796257972717285
1.3796257972717285
1.4378615617752075
1.4378615617752075
1.422987699508667
1.422987699508667
1.408454179763794
1.408454179763794
1.4041879177093506
1.4041879177093506
1.4276624917984009
1.4276624917984009
1.4458825588226318
1.4458825588226318
1.5070369243621826
1.5070369243621826
1.4582675695419312
1.4582675695419312
1.4353320598602295
1.4353320598602295
1.4360344409942627
1.4360344409942627
1.4060547351837158
1.4060547351837158
1.3721834421157837
1.3721834421157837
1.4859342575073242
1.4859342575073242
1.4752308130264282
1.4752308130264282
1.5029387474060059
1.5029387474060059
1.5103423595428467
1.5103423595428467
1.4800370931625366
1.4800370931625366
1.3849263191223145
1.3849263191223145
1.436884880065918
1.436884880065918
1.417528748512268
1.417528748512268
1.492363691329956
1.492363691329956
1.4123766422271729
1.4123766422271729
1.4054125547409058
1.4054125547409058
1.4886075258255005
1.4886075258255005
1.4721190929412842
1.4721190929412842
1.4615099430084229
1.4615099430084229
1.4320951700210571
1.4320951700210571
1.4198548793792725
1.4198548793792725
1.417236089706421
1.417236089706421
1.4173119068145752
1.4173119068145752
1.4298810958862305
1.4298810958862305
1.3706982135772705
1.3706982135772705
1.4235960245132446
1.4235960245132446
1.3855185508728027
1.3855185508728027
1.4296832084655762
1.4296832084655762
1.4478527307510376
1.4478527307510376
1.3492720127105713
1.3492720127105713
1.494626522064209
1.494626522064209
nan
nan
nan
nan
nan
nan
nan

Actually I have regenerated the input data, and make sure none of them has a sum == 0. Now I am really confused what could be wrong with my model or data.