Answers.Body: <p>Fixed. Was an issue with the random generator. In my class for the Neuron layer where I initialize the weights I get new doubles from the generator for each of the initial weight values, but I found a bug where I was re-initializing the random generator, which was of course causing different values.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-15T16:51:27.173Z
Answers.Text: If you feel inspired (and want the rep) it would be helpful if you could elaborate and accept your answer.
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-10T11:15:46.327Z
Answers.LastActivityDate: 2018-10-17T04:05:46.387Z
Answers.LastEditDate: 2018-10-17T04:05:46.387Z
Answers.UserDisplayName: MikeB
Body: <p>I have written my own basic convolutional neural network in Java as a learning exercise. I am using it to analyze the MIT CBCL face database image set. They are a set of 19x19 pixel greyscale images.</p>
  
  <p>Network specifications are:</p>
  
  <p>Single Convolution Layer with 1 filter:
  Filter Size: 4x4.
  Stride Size: 1</p>
  
  <p>Single Pooling Layer
  2x2 Max Pooling</p>
  
  <p>3 layer MLP(input, 1 hidden and output)
  input = 64 neurons
  hidden = 15 neurons
  output = 2 neurons
  learning rate = 0.1</p>
  
  <p>Now I am getting reasonable accuracy(92.85%), but my issue is that it is being achieved at very different points in the epoch count across network runs:</p>
  
  <pre><code>Epochs  Training Accuracy   Test Accuracy   Validation Accuracy
  </code></pre>
  
  <p>Run 1   415 93.13   92.44   93.35
  Run 2   515 92.44   93.18   92.84
  Run 3   327 93.83   92.05   92.38</p>
  
  <p>I am using the Java random class with the same seed for every run to initialize the kernel, the MLP weights and break the input data into 3 sets.(training is being done using the 33-33-33 method)</p>
  
  <p>I am a loss as to what is causing this variation in epoch count to achieve the highest point in validation accuracy. Can anybody explain this?</p>
  
Tags: <convolutional-neural-networks><java>
Title: Huge variations in epoch count for highest generalized accuracy in CNN
UserDisplayName: MikeB
ThreadText: # Huge variations in epoch count for highest generalized accuracy in CNN
  
  (Asked by: username_0 on 2018-10-07)
  
  I have written my own basic convolutional neural network in Java as a learning exercise. I am using it to analyze the MIT CBCL face database image set. They are a set of 19x19 pixel greyscale images.
  
  Network specifications are:
  
  Single Convolution Layer with 1 filter:
  Filter Size: 4x4.
  Stride Size: 1
  
  Single Pooling Layer
  2x2 Max Pooling
  
  3 layer MLP(input, 1 hidden and output)
  input = 64 neurons
  hidden = 15 neurons
  output = 2 neurons
  learning rate = 0.1
  
  Now I am getting reasonable accuracy(92.85%), but my issue is that it is being achieved at very different points in the epoch count across network runs:
  
  ```
  Epochs  Training Accuracy   Test Accuracy   Validation Accuracy
  
  ```
  
  Run 1 415 93.13 92.44 93.35
  Run 2 515 92.44 93.18 92.84
  Run 3 327 93.83 92.05 92.38
  
  I am using the Java random class with the same seed for every run to initialize the kernel, the MLP weights and break the input data into 3 sets.(training is being done using the 33-33-33 method)
  
  I am a loss as to what is causing this variation in epoch count to achieve the highest point in validation accuracy. Can anybody explain this?
  
  **Tags:** convolutional-neural-networks, java
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2018-10-10. Score: 1 | Accepted answer
  
  Fixed. Was an issue with the random generator. In my class for the Neuron layer where I initialize the weights I get new doubles from the generator for each of the initial weight values, but I found a bug where I was re-initializing the random generator, which was of course causing different values.
  
  > **Comments:**
  > **username_1**: If you feel inspired (and want the rep) it would be helpful if you could elaborate and accept your answer.  
  
  ---
  
  