Your targets would have to be already multi-hot encoded. If your current target shape is [batch_size] and the targets contain values in [0, nb_classes-1], then they are multi-class classification targets and you could use nn.CrossEntropyLoss as described before.
However, if you want to use these targets for a multi-label classification for specific reasons, you could use F.one_hot to create the expected shape for nn.BCEWithLogitsLoss.