If I change this part in the training function,
for j,((input_a, labels_a),(input_b, labels_b)) in enumerate(zip(loaders[split][0],loaders[split][1])):

            input_a = input_a.to(dev)
            input_b = input_b.to(dev)

            labels_a = labels_a.long().to(dev)
            labels_b = labels_b.long().to(dev)
            labels_a = labels_a.squeeze()
            labels_b = labels_b.squeeze()
            #labels_a = labels_a.unsqueeze(1)
            #labels_b = labels_b.unsqueeze(1)

            #labels_a = labels_a.argmax(-1)
            #labels_b = labels_b.argmax(-1)

            inputs = torch.cat([input_a,input_b],axis=0)
            labels = torch.cat([labels_a, labels_b])

            labels  = labels.squeeze()
            #labels = labels.argmax(-1)

I get this error:
RuntimeError: The size of tensor a (10) must match the size of tensor b (512) at non-singleton dimension 1