Sorry for the late reply.
If I understand correctly, you want to consider the neighbors of the target when computing the loss. nn.CrossEntropyLoss does not permit that, but I implemented a custom loss function that takes into account the 2 neighbors of the current target.
class CustomLoss(nn.Module):
    def __init__(self, weights):
        super(CustomLoss, self).__init__()
        self.weights = weights

    def forward(self, logits, targets):
        loss = torch.tensor([0], dtype=torch.float32)
        log_probabilities = F.log_softmax(logits)
        for i in range(targets.shape[0]):
            low = int(max(0, targets[i] - 1))
            high = int(min(log_probabilities.shape[1], targets[i] + 2))
            p = log_probabilities[i, low:high]
            loss += -torch.sum(torch.mul(p, self.weights[(low - low):(high - low)]))
        return loss

we = torch.tensor([0.2, 0.5, 0.2], dtype=torch.float32)
criterion1 = CustomLoss(we)

So basically, the target has a weight of 0.5 while its two neighbors (if they both exists) have weight of 0.2.
Let me know if it helps!