Body: <p>I have been trying to write code to implement plain neural net without convolution from scratch. I took some help online <a href="https://towardsdatascience.com/building-neural-network-from-scratch-9c88535bf8e9" rel="nofollow noreferrer">here</a> and added my code to my github <a href="https://github.com/shekh01ml/mnist_ann.git" rel="nofollow noreferrer">account</a>.</p>
  
  <p>I don't understand why the prediction made by my code is only 88%-90% accurate after the 1st epoch, whereas his code is 95% accurate after 1st epoch with the same parameters (Same Xavier initialization for weights, biases are not initialized, same hidden layer neurons). While his architecture uses 2 hidden layers, my code performed worse with 2 hidden layers. For 1 hidden layer, his code performs similar (~96%). </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-03T19:45:18.157Z
Comments.Text: Different starting weights or ordering of the training data might make a difference in the first epoch. Do the two programs perform similarly after they've been given a chance to converge? And do you see similar differences in performance when you rerun your code with a different random seed?
Comments.UserDisplayName: Ray
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-03T22:05:57.340Z
Comments.Text: No my program only has around 90-92% accuracy even after 30 epochs (with 200 neurons in hidden layer), and performs similarly on different seeds. It only reaches 93% on first epoch when i increase hidden layer neurons to 1000.
Comments.UserDisplayName: shekhar
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-04T08:14:26.393Z
Comments.Text: Ok I found the mistake. The weights and biases of the input to hidden layer were not getting updated.
Comments.UserDisplayName: shekhar
Tags: <neural-networks><machine-learning><image-recognition>
Title: MNIST Classification code performing with 88%-90% whereas other codes online perform 95% on first epoch
UserDisplayName: shekhar
ThreadText: # MNIST Classification code performing with 88%-90% whereas other codes online perform 95% on first epoch
  
  (Asked by: username_0 on 2020-05-03)
  
  I have been trying to write code to implement plain neural net without convolution from scratch. I took some help online [here](https://towardsdatascience.com/building-neural-network-from-scratch-9c88535bf8e9) and added my code to my github [account](https://github.com/shekh01ml/mnist_ann.git).
  
  I don't understand why the prediction made by my code is only 88%-90% accurate after the 1st epoch, whereas his code is 95% accurate after 1st epoch with the same parameters (Same Xavier initialization for weights, biases are not initialized, same hidden layer neurons). While his architecture uses 2 hidden layers, my code performed worse with 2 hidden layers. For 1 hidden layer, his code performs similar (~96%).
  
  **Tags:** neural-networks, machine-learning, image-recognition
  
  > **Comments:**
  > **username_1**: Different starting weights or ordering of the training data might make a difference in the first epoch. Do the two programs perform similarly after they've been given a chance to converge? And do you see similar differences in performance when you rerun your code with a different random seed?  
  > **username_0**: No my program only has around 90-92% accuracy even after 30 epochs (with 200 neurons in hidden layer), and performs similarly on different seeds. It only reaches 93% on first epoch when i increase hidden layer neurons to 1000.  
  > **username_0**: Ok I found the mistake. The weights and biases of the input to hidden layer were not getting updated.  
  
  ---
  
  ## 0 answers
  
  