You are right! It seems I was just lucky to have used another seed (in fact running the code line by line a few times) and it seems your parameter initialization might be quite sensitive.
If you set the seed to 2809 you should get valid results.
However, I would recommend to try to stabilize the training.