Dear pytorch guys  ,
first of all thanks for developing such a conceptually smart, easy to use and nice ML framework.
I am a pytorch user since the first versions and I have few questions about how the NN losses are conceptually implemented:


BCE and BCEWithLogitLoss require Float labels (also called targets if you prefer). Why is that, precisely? The CrossEntropyLoss, for example, requires Long labels (integers) and it’s a more general case than BCE, so why the BCE* need floats? This choice influences also the second point I am going to raise


BCE and BCEWithLogit don’t (currently) support ignore_index, which is IMO an extremely useful feature. Implementing the ignore_index is easy, but the fact that these losses take in Float label make it a bit awkward to do, because due to numeric problems, the float representation of the ignore_index value could be very similar but different enough to fail the


mask = labels.ne(ignore_index)
clause that I could use to implement such a feature. (e.g. -1.0000000001 instead of -1).

Minor issue: in the pytorch 1.1.0 BCEWithLogitLoss documentation, I think that there is a problem in the pos_weight example: the (10,64) tensor is said to contain 10 batches of 64 classes, but from what I see, if there are 64 classes, then 10 must be the samples (with batch omitted). Conversely, if 10 is the batch size, the 64 should be the samples per batch (with only 1 binary classification going on).

To wrap up my comments:
Would it then be possible maybe to uniform a bit more the “interface” of the losses functions, ensuring that:
-losses for classification use integer labels/targets
-all the losses for classification provide ignore_index
-all the losses for classification provide class_weights
thanks in advance and keep up with the good work!
cheers,
Daniele