Answers.Body: <p>The trick was to normalize the input dataset values with the respective mean and standard deviation in each column. This reduced the loss drastically, and my network is training more efficiently now. Moreover, normalizing the data also helps you calculate the weights associated with each input node more easily, especially when trying to find out variable importance.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-27T04:52:05.610Z
Answers.LastActivityDate: 2020-01-27T04:52:05.610Z
Answers.UserDisplayName: Prishita Ray
Body: <p>I am training an undercomplete autoencoder network for feature selection. I am using one hidden layer in the encoder and decoder networks each. The ELU activation function is used for each layer. For optimization, I am using the ADAM optimizer. Moreover, to improve convergence, I have also introduced learning rate decay. The model shows good convergence initially, but later starts to generate losses (12 digits) in the same range of values for several epochs, and is not converging. How to solve this issue?</p>
  
Tags: <autoencoders><learning-rate>
Title: Autoencoder network for feature selection not converging
UserDisplayName: Prishita Ray
ThreadText: # Autoencoder network for feature selection not converging
  
  (Asked by: username_0 on 2020-01-23)
  
  I am training an undercomplete autoencoder network for feature selection. I am using one hidden layer in the encoder and decoder networks each. The ELU activation function is used for each layer. For optimization, I am using the ADAM optimizer. Moreover, to improve convergence, I have also introduced learning rate decay. The model shows good convergence initially, but later starts to generate losses (12 digits) in the same range of values for several epochs, and is not converging. How to solve this issue?
  
  **Tags:** autoencoders, learning-rate
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2020-01-27. Score: 1 | Accepted answer
  
  The trick was to normalize the input dataset values with the respective mean and standard deviation in each column. This reduced the loss drastically, and my network is training more efficiently now. Moreover, normalizing the data also helps you calculate the weights associated with each input node more easily, especially when trying to find out variable importance.
  
  
  ---
  
  