Hi Thomas -



 thomas:

The pytorch loss doesn’t seem to train the network


The short answer is that your pytorch code only takes one optimizer
step.


Tensorflow code:
model.fit(np.ones((1000,10), dtype='float'),np.asarray(1000*[[0.7, 0.3, 0.0, 0.0]], dtype='float'))



With default arguments, tf.keras.Model.fit() performs one epoch
of training with a batch size of 32.  You pass in 1000 samples, so you
will train for about 30 optimizer steps.  This should be enough to train
your very simple model, at least to some degree.
(Note, I have not looked at the rest of your code in any detail.)


Pytorch code:
optimizer.step()



In contrast, torch.optim.SGD.step() takes only a single step of the
optimizer.  In pytorch, you have to write your own (very simple) optimizer
loop.
Good luck.
K. Frank