Body: <p>I want to initialize a parameter, which is a single real number in my model. If you want the role of the parameter in the model, you can assume it as the parameter to multiply with the output of the neural network and the resultant product will be the final output.</p>
  <p>Does initialization value matter? If yes, are there any guidelines on initializing the single parameter?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-08T02:25:16.550Z
Comments.Text: Initialization always matters. There's countless papers on parameter initialization, however a common method (last I checked) is to use [Kaiming initialization](https://towardsdatascience.com/weight-initialization-in-neural-networks-a-journey-from-the-basics-to-kaiming-954fb9b47c79). In this case since it's only one parameter it should be fairly simple to initialize using this method
Comments.UserDisplayName: Recessive
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-10T09:03:19.183Z
Comments.Text: In this case, the most suitable initialization may also depend on the _specific_ role of this parameter. You say you multiply it with the output of the model, but why are you doing this? Maybe you can provide more details about that.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-11T02:44:39.867Z
Comments.Text: Ha @nbro I will try to do.
Comments.UserDisplayName: hanugm
Tags: <deep-learning><weights><weights-initialization>
Title: Are there any recommendations on initialising a single parameter in deep learning?
UserDisplayName: hanugm
ThreadText: # Are there any recommendations on initialising a single parameter in deep learning?
  
  (Asked by: username_0 on 2021-12-07)
  
  I want to initialize a parameter, which is a single real number in my model. If you want the role of the parameter in the model, you can assume it as the parameter to multiply with the output of the neural network and the resultant product will be the final output.
  
  Does initialization value matter? If yes, are there any guidelines on initializing the single parameter?
  
  **Tags:** deep-learning, weights, weights-initialization
  > **Comments:**
  > **username_1**: Initialization always matters. There's countless papers on parameter initialization, however a common method (last I checked) is to use [Kaiming initialization](https://towardsdatascience.com/weight-initialization-in-neural-networks-a-journey-from-the-basics-to-kaiming-954fb9b47c79). In this case since it's only one parameter it should be fairly simple to initialize using this method  
  > **username_2**: In this case, the most suitable initialization may also depend on the _specific_ role of this parameter. You say you multiply it with the output of the model, but why are you doing this? Maybe you can provide more details about that.  
  > **username_0**: Ha @nbro I will try to do.  
  
  ---
  
  ## 0 answers
  
  