As others have noted, the non-linearity between the two linear layers is essential. The composition of two linear layers is linear, but XOR cannot be computed with a linear function.
To get your network to learn XOR every time, you can do the following:


Change your criterion to MSELoss. Unless you know exactly why you need it, don’t use L1Loss, since it usually requires more careful tuning of hyper-parameters.


Change your optimizer to Adam. SGD requires more careful tuning of hyper-parameters and weight initialization. Adam with a learning rate between 0.001 and 0.0001 generally works well. In this case, I found that 0.001 converges fast.