Hi atl!



 atl:

the accuracy drops from 50% to 3% during the course of training:


It sounds like you are calculating your accuracy backwards.
If you’re getting an accuracy of 3% on a binary problem, then you
could just as well be making predictions with 97% accuracy by
swapping your “0” predictions for “1” and vice versa.
Is your loss decreasing sensibly as you train?


I’m still a bit confused though about how I should initialise my inputs, so that the normal distributions are not the same for y=0 and y=1.


The way you construct x should work, but it would be a lot cleaner
(and more efficient) if you would get rid of the for and if statements.
You can do it all with pytorch tensor operations, which, in general, is
to be preferred.
I think I would do something like:
x_m = (1 - y) * torch.normal (-1.0, 1.3, size = (10000, 1)) + y * torch.normal (1.0, 1.3, size = (10000, 1))
x_c = (1 - y) * torch.normal (-1.0, 0.1, size = (10000, 1)) + y * torch.normal (1.0, 0.1, size = (10000, 1))
x = torch.cat ((x_m, x_c), 1)

This has the minor disadvantage that you generate twice as many
random samples as you need, throwing half away, but it looks simpler
and easier to read to me than the fancier (and more efficient)
alternatives.
As an aside, notice that for the second of your two input values
(the x_c piece), the means of your two distributions (that is, for
y = 0 and y = 1) differ by about fifteen standard deviations,
having in practice no overlap.  Therefore it should be very easy
to construct a perfect classifier.  After you get this working, it might
be fun to make the standard deviation of your x_c distributions
something like 1.0 so that the y = 0 and y = 1 distributions do,
in fact,  overlap, and see how large an accuracy you can achieve
in practice with your network vs. the maximum theoretical accuracy.
Best.
K. Frank