Unfortunately, no, that’s not what I’m looking to do. Using your example as a start:
model = nn.Linear(10, 2)
criterion = nn.CrossEntropyLoss()
opt = AdamOptimizer(model.parameters())
x = torch.randn(1, 10)
target = torch.empty(1, dtype=torch.long).random_(2)
losses = 0
weights = torch.ones(10, requires_grad=True)
for _ in range(10):
    output = model(x)
    loss = criterion(output, target, reduction='none')
    w_loss = (loss * weights).sum() / weights.sum()
    losses += w_loss
print(losses)
losses.backward()
opt.step()

weights.grad.data.zero_()
model.zero_grad()
for _ in range(10):
    output = model(x)
    loss = criterion(output, target)
    loss.backward()
## now I would like the gradient with respect to the weights (but I find that it is 0)