I print their size out and their size is:

torch.Size([210226, 128])
torch.Size([235481, 128])
torch.Size([210226])
torch.Size([235481])

noting that the computation is parallelized on two GPUs, so the pred and lbl size on each GPU is:
torch.Size([210226, 128])
torch.Size([210226])
and
torch.Size([235481, 128])
torch.Size([235481])
The most confusing thing is when I cancel the parallelization and run it on a single GPU, it works perfectly.