Solved by ptrblck in post #30 
              
              
                You could split the outputs into the regression problem (mouse coordinates) and the classification problem (click/no-click). 
Both outputs should be passed to the appropriate loss function. 
Here is a very simple example code you could use as a starter: 
class MyModel(nn.Module):
    def __init__(se…