Answers.Body: <p>There is nothing wrong with changing your cost/loss function after every step while training a neural network. For example, this paper looks at a weight decay scheduler (which changes the weight regularization at every step): <a href="https://arxiv.org/abs/2006.08643" rel="nofollow noreferrer">https://arxiv.org/abs/2006.08643</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-15T13:39:22.447Z
Answers.LastActivityDate: 2023-04-15T13:39:22.447Z
Answers.UserDisplayName: shatz
Body: <p>I have a neural network that is being trained with a changing cost function. Could I use backpropagation at all? If yes, how would I do this?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-15T09:29:54.347Z
Comments.Text: Hi, I think we need more detail to answer this question. What do you mean by "changing cost function"? Are some of your NN parameters influenced by an external stochastic variable? Then you'd have to look in to Monte Carlo methods for gradient estimation, but it's really hard to say whether that is what you are looking for.
Comments.UserDisplayName: postnubilaphoebus
Tags: <backpropagation>
Title: How would I use backpropagation with a changing cost function
UserDisplayName: 922073
ThreadText: # How would I use backpropagation with a changing cost function
  
  (Asked by: username_0 on 2023-04-14)
  
  I have a neural network that is being trained with a changing cost function. Could I use backpropagation at all? If yes, how would I do this?
  
  **Tags:** backpropagation
  > **Comments:**
  > **username_1**: Hi, I think we need more detail to answer this question. What do you mean by "changing cost function"? Are some of your NN parameters influenced by an external stochastic variable? Then you'd have to look in to Monte Carlo methods for gradient estimation, but it's really hard to say whether that is what you are looking for.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-04-15. Score: 0 | Accepted answer
  
  There is nothing wrong with changing your cost/loss function after every step while training a neural network. For example, this paper looks at a weight decay scheduler (which changes the weight regularization at every step): <https://arxiv.org/abs/2006.08643>.
  
  
  ---
  
  