My provisional fix:
example training loop:

for epoch in range(EPOCHS):
    for i, (images, labels) in tqdm(enumerate(train_dl)):
        images = images.to(device)
        labels = torch.eye(3)[labels].to(device)
        
        preds = model(images)
        loss = loss_func(preds, labels)

        optimizer.zero_grad()
        loss.backward()
        optimizer.step()


During every iteration, when pushing the labels to the device (‘cuda’ or ‘cpu’ etc.), a one hot vector is created.
labels = torch.eye(3)[labels].to(device)
torch.eye(3)

'''output:

[[1, 0, 0]
 [0, 1, 0]
 [0, 0, 1]]

'''

torch.eye(3)[0]

'''output:

[1, 0, 0]

'''

So when the label is 0 the one hot vector is [1, 0, 0], for 1 it is [0, 1, 0] and so on.
Still, I’d like a fix using the target_transform attribute tho!