@fmassa, Are you replying to point 3?
MultiLabelSoftMarginLoss is already there and I’m aware of it. It’s implemented in terms of binary_cross_entropy + sigmoid.
As far as I know doing logsigmoid (or logsoftmax) in one go is:

faster than doing sigmoid then taking the log (log is done within binary_cross_entropy)
numerically stabler

Hence my question if there is a corresponding loss function to logsigmoid as there is for logsoftmax (NLLLoss)