Hi atl!



 atl:

The network doesn’t seem to learn
for t in range(batchlen):
    if y[t] == 0:
        x[t] = torch.randn(1)*(-1)
    elif y[t] == 1:
        x[t] = torch.randn(1)



The x you generate for y = 0 isn’t really different than that for  y = 1.
torch.randn() draws samples from the normal distribution, and
this distribution is symmetrical about zero.  Therefore randn() and
randn() * (-1) are statistically the same.
That is, your inputs for y = 0 and y = 1 aren’t actually any different,
so your network can’t learn.
As a second issue, you aren’t calculating your accuracy correctly.


def binary_acc(pred, target):
    pred_tag = torch.round(torch.sigmoid(pred))

    correct_results_sum = (pred_tag == target).sum().float()
    acc = correct_results_sum/target.shape[0]
    acc = torch.round(acc * 100)
    
    return acc



Here, you are applying sigmoid() twice in your accuracy calculation,
once in the last “layer” of your model:


    self.softmax = nn.Sigmoid()
...
    x = self.softmax(x)



(Calling your last layer softmax when it’s really nn.Sigmoid() is a
confusing naming choice.)
and then again in your binary_acc() function.
So the first sigmoid() (in model) takes the output of
self.output = nn.Linear(100,1) that ranges from -inf to inf
and maps it to [0.0, 1.0].  The second sigmoid() (in binary_acc())
maps this to [0.5, 1.0].  This is then always rounded to 1, so you
compare 100% 1’s in pred_tag to 50% 1’s in target and return
50% for your accuracy, acc.
(As an aside, best practices would recommend, for reasons of
numerical stability, that you get rid of the final Sigmoid in model
and use BCEWithLogitsLoss(rather thanBCELoss`) as your
loss function.)
Good luck.
K. Frank