I tried both of your proposal, but with no success. Although the loss changes, the pattern still remains the same across all epochs. It doesn’t matter if I don’t use nn.Sigmoid() as the last activation function and use nn.BCEWithLogitsLoss or no sigmoid and nn.BCELoss. Though I did not really understand how to incorporate what @pascal_notsawo suggested. I tried:
pred = model(X)
pred = torch.sigmoid(pred).round()
loss = loss_fn(pred, y)

This is the training performance for Sigmoid activation with BCELoss, also the accuracy of 1500% indicates that something is really wrong:
Epoch 1
-------------------------------
loss: 11.000000  [    0/  600]
loss: 21.000000  [  100/  600]
loss: 25.000000  [  200/  600]
loss: 22.000000  [  300/  600]
loss: 19.000000  [  400/  600]
loss: 14.000000  [  500/  600]
Test Error: Accuracy: 1500.0%, Avg loss: 0.150000 
...
Epoch 5
-------------------------------
loss: 11.000000  [    0/  600]
loss: 21.000000  [  100/  600]
loss: 25.000000  [  200/  600]
loss: 22.000000  [  300/  600]
loss: 19.000000  [  400/  600]
loss: 14.000000  [  500/  600]
Test Error: 
 Accuracy: 1500.0%, Avg loss: 0.150000

When I monitor the output of the model during training, I see that it correctly outputs a Tensor of shape [batch_size, 1] containing 0 or 1. I don’t really understand this output, as I use the sigmoid activation I would expect a number between 0 and 1 as output, but not exclusively 0 or 1.
I further observed that the output generated by the model for the last batch in epoch 1 is identical to the output generated for the same batch in the last epoch. (I collected the outputs and compared the tensors after training) This seems to indicate to me, that the model does not update its parameters.
So my actual question is at the moment, what am I doing wrong that the model does not output values between 0 and 1, but 0 or 1? If it would, using nn.BCELoss should be the right way to go. The same model architecture with BCELoss works btw in Keras.
I’d be open to suggestion on how to change the architecture of the model to make it work; my data looks as following and the task is to predict the label of loan_status (0 or 1). (The data is not completely cleaned and preprocessed yet)
df.head()

Expand Table



loan_status
int_rate
installment
annual_inc
loan_amnt
dti
open_acc
pub_rec
revol_bal
revol_util
…
seniority
term
emp_length
home_ownership
verification_status
purpose
initial_list_status
application_type
address
issue_d




0
1
-0.491799
-0.408291
117000.0
-0.492243
1.194178
0.913445
-0.34997
1.340611
-0.491258
…
0.357143
-1
0
2
-1
10
1
1
22690
2015


1
1
-0.368816
-0.662750
65000.0
-0.731551
0.646367
1.108239
-0.34997
-0.174517
-0.019706
…
0.157143
-1
1
0
-1
2
-1
1
5113
2015


2
1
-0.704225
0.299609
43057.0
0.177819
-0.564308
0.329061
-0.34997
-0.813871
1.575371
…
0.114286
-1
1
2
1
1
-1
1
5113
2015


3
1
-1.598649
-0.842348
54000.0
-0.827274
-1.896573
-1.034500
-0.34997
-0.495024
-1.323650
…
0.114286
-1
1
2
-1
1
-1
1
813
2014


4
0
0.811824
0.707861
55000.0
1.227783
1.685770
0.329061
-0.34997
-0.465887
0.656869
…
0.200000
1
1
0
1
1
-1
1
11650
2013