Answers.Body: <p>I found that the peak was caused by the data I am using. Specifically, the MinMaxScaler changed the data shape and I resolved the issue by simply dividing to the max value.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-24T18:49:35.863Z
Answers.LastActivityDate: 2019-12-24T18:49:35.863Z
Answers.UserDisplayName: Cristian M
Body: <p>I am training a simple convolutional neural network to recognize two types of 1024-point frequency spectra (FFT). This is the model I'm using:</p>
  
  <pre><code>cnn = Sequential()
  cnn.add(Conv1D(filters=64, kernel_size=3, activation=LeakyReLU(), input_shape=(nInput,1)))
  cnn.add(Conv1D(filters=64, kernel_size=3, activation=LeakyReLU()))
  cnn.add(MaxPooling1D(pool_size=2))
  cnn.add(Flatten())
  cnn.add(Dense(nFinalDense, activation=LeakyReLU()))
  cnn.add(Dense(nOutput, activation='sigmoid'))
  </code></pre>
  
  <p>However I get the following accuracy and loss during training:
  <a href="https://i.sstatic.net/eXd1x.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/eXd1x.png" alt="enter image description here"></a></p>
  
  <p>Why do I get the large peak in both plots? How can it be explained? Is there a problem with the data I'm using (I mention that I obtain a similar peak when training an autoencoder for denoising using the same data)?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-24T13:36:38.423Z
Comments.Text: @DuttaA I get it multiple times. I'm using Tensorflow to train the model with default weight initializers for each layer, so it automatically randomizes the weights. I mention that I set the seed values for numpy and tensorflow at the beginning of the script to get a more deterministic output. I've changed the seed value and I get a smaller peak at a different epoch. What do you think it is?
Comments.UserDisplayName: Cristian M
Tags: <convolutional-neural-networks><training><accuracy><loss>
Title: How to explain peak in training history of a convolutional neural network?
UserDisplayName: Cristian M
ThreadText: # How to explain peak in training history of a convolutional neural network?
  
  (Asked by: username_0 on 2019-12-24)
  
  I am training a simple convolutional neural network to recognize two types of 1024-point frequency spectra (FFT). This is the model I'm using:
  cnn = Sequential()
  cnn.add(Conv1D(filters=64, kernel_size=3, activation=LeakyReLU(), input_shape=(nInput,1)))
  cnn.add(Conv1D(filters=64, kernel_size=3, activation=LeakyReLU()))
  cnn.add(MaxPooling1D(pool_size=2))
  cnn.add(Flatten())
  cnn.add(Dense(nFinalDense, activation=LeakyReLU()))
  cnn.add(Dense(nOutput, activation='sigmoid'))
  
  However I get the following accuracy and loss during training:
  
  Why do I get the large peak in both plots? How can it be explained? Is there a problem with the data I'm using (I mention that I obtain a similar peak when training an autoencoder for denoising using the same data)?
  
  **Tags:** convolutional-neural-networks, training, accuracy, loss
  > **Comments:**
  > **username_0**: @DuttaA I get it multiple times. I'm using Tensorflow to train the model with default weight initializers for each layer, so it automatically randomizes the weights. I mention that I set the seed values for numpy and tensorflow at the beginning of the script to get a more deterministic output. I've changed the seed value and I get a smaller peak at a different epoch. What do you think it is?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2019-12-24. Score: 1 | Accepted answer
  
  I found that the peak was caused by the data I am using. Specifically, the MinMaxScaler changed the data shape and I resolved the issue by simply dividing to the max value.
  
  
  ---
  
  