I’m not sure, if I understand the use case correctly, but would scattering the output logits using a mapping work?
Here is a small example:
batch_size = 10
nb_classes = 2
nb_features = 5
nb_out = 10

x = torch.randn(batch_size, nb_features)
target = torch.randint(0, nb_classes, (batch_size,))

mapping = torch.tensor([0, 0, 0, 0, 0, 1, 1, 1, 1, 1])

model = nn.Linear(nb_features, nb_out)
criterion = nn.CrossEntropyLoss()

output = model(x)
output_small = torch.zeros(batch_size, nb_classes).scatter_add(1, mapping.unsqueeze(0).expand_as(output), output)
loss = criterion(output_small, target)
loss.backward()
print(model.weight.grad)