In the meantime, I’ve solved this by iterating over the batch. Still curious if there exists a more elegant way.
>>>def to_indices(batch, padding_value=-1):
   ...:     indices = []
   ...:     for sample in batch:
   ...:         indices.append(sample.nonzero(as_tuple=True)[0])
   ...:     return nn.utils.rnn.pad_sequence(indices, batch_first=True, padding_value=padding_value)
>>> batch
tensor([[1., 0., 0., 1., 1.],
        [0., 1., 0., 0., 0.],
        [0., 1., 1., 0., 0.]])
>>> to_indices(batch)
tensor([[ 0,  3,  4],
        [ 1, -1, -1],
        [ 1,  2, -1]])