Solved by ptrblck in post #2 
              
              
                You might want to play around with some hyperparameters and e.g. lower the learning rate to ~1e-3, as I cannot see any obvious errors in your code.