Yes, that is what I said that is an option but that does not work well for longer options. If one token corresponds to one option, then you can do this - but if you want your code to work dynamically (also with larger labels that are tokenised with multi tokens), then this won’t work.
By the way, using torch.zeros(logits.shape[-1], device="cuda") will probably be faster.
I am curious to see whether @valhalla knows how to deal with this issue in a better, general way.