keep the output of your network to be coming from the Linear layer. Then i you want binary / multi-label, use the functions: BCEWithLogitsLoss or CrossEntropyLoss which both compute the logits internal to their formulation