Solved by ptrblck in post #2 
              
              
                If you want to reduce the model capacity, you could lower the number of hidden neurons in the model. 
Also note that nn.BCELoss is expecting a single neuron with a sigmoid applied on it for a binary classification use case. 
So you would have to change the last linear layer to nn.Linear(512, 1) and …