Thank you for your explanation.
I am using MNIST data for multi-class classification (there are ten classes, 0, 1 through 9).
I could use both nn.CrossEntropyLoss() and nn.BCEWithLogtisLoss() on MNIST and get good performance. However, before reading your reply, I thought nn.BCEWithLogitsLoss() could only be using in the multi-label classification (you said it can be used in this setting).
Then here comes two questions


Focusing on multi-class classification. If using two losses can both achieve good performance, I suspect they should be similar in some sense but they actually use Softmax and multiple Sigmoid respectively, which I deem very different. For example, for Softmax, it makes sense to choose the class that has highest probability. However, for Sigmoid, it likely that two or more classes have very close output (say 0.79, 0.8, 0.81), then choosing one particular class does not seem to make sense and this is main confusion for using nn.BCEWithLogitsLoss() for multi-class classification.

Focusing on multi-label classification. Set nn.CrossEntropyLoss() aside and we now focus on nn.BCEWithLogitsLoss().  If I am building an image recognition system that is trying to identify whether there are car, people and stop sign in the given image at the same time. Now it is clear this is a multi-label classification problem (with three labels). However, people generally set 0.5 as a decision threshold for 1/0 classification. So a output like [0.2, 0.3, 0.4] will all be regarded as [0, 0, 0] by hard thresholding. But I believe this is similar to Logistic regression and the decision threshold should be tuned and using hard thresholding does not make sense. Now comes the main question. How do I tune three thresholds at the same time or maybe I just output probability (like many object detection algorithms do).