Background
-I took a working Python code for sample neural net( analtically derive gradient, Victor Zhou Blog)
-I converted it into PyTorch with autograd
-My aim is to explain NN through Pytorch & GPU/CUDA - to beginners
-I have hard coded weights to make the compute deterministic
-CPU version
-Python VERSION: 3.8.3
-pyTorch VERSION: 1.5.1
Problem
-Though Python code converges, pytorch code terminates with Error & NAN
-Am I making any basic mistakes?( I am new to PyTorch)
-since pure Python version works fine, why torch code explodes with NAN?
after running for 80 epochs
PyTorch code given below:

  
      gist.github.com
  
  
    https://gist.github.com/snsmssss/0369052c6daf7a001660e51ba9da9e92 4
gistfile1.txt
import torch
def sigmoid(x):
  # Sigmoid activation function: f(x) = 1 / (1 + e^(-x))
  sx =   1.0 / (1.0 + torch.exp(-x))
  return sx 

# y_true and y_pred are numpy arrays of the same length.
def mse_loss(y_true, y_pred):
  return torch.mean((y_true - y_pred) ** 2)

This file has been truncated. show original




  
  
    
    
  
  


OUTPUT

  
      gist.github.com
  
  
    https://gist.github.com/snsmssss/0e6f4e49de6af9d5ca2a8cb7bf2b0260 3
Output with Error message
Epoch 0 loss: 0.334
Epoch 10 loss: 0.305
Epoch 20 loss: 0.118
Epoch 30 loss: 0.072
Epoch 40 loss: 0.049
Epoch 50 loss: 0.047
Epoch 60 loss: 0.046
Epoch 70 loss: 0.037
Epoch 80 loss: 0.024
---------------------------------------------------------------------------
This file has been truncated. show original