Body: <p>A loss function is a measure of how bad our neural network is. We can decrease the loss by proper training.</p>
  <p>I came across the phrase &quot;adaptive loss function&quot; in several research papers. For example: consider the following excerpt from the &quot;Introduction&quot; of the research paper titled <a href="https://proceedings.mlr.press/v48/reed16.pdf" rel="nofollow noreferrer">Generative Adversarial Text to Image Synthesis</a> by Scott Reed et al.</p>
  <blockquote>
  <p>By conditioning both generator and discriminator on side information, we can naturally model this phenomenon since the discriminator network acts as a &quot;smart&quot; <strong>adaptive</strong> loss function.</p>
  </blockquote>
  <p>When can we denote a loss function as adaptive? Is it a mathematical property or is solely based on the context?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-03T09:48:40.783Z
Comments.Text: It is the normal English meaning of "adaptive".
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-03T09:58:14.403Z
Comments.Text: @user253751 So, if the loss function is changing according to the need, then can I say it as adaptive?
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-03T09:58:47.460Z
Comments.Text: Sure. It is the normal English meaning. It is not something with a precise mathematical definition, as far as I'm aware. AI papers are not a programming language.
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-03T10:17:44.143Z
Comments.Text: Did you come across the term in context other than GANs? I can see how the discriminator might be considered to present an "adaptive" loss function for GANs as user253751 states. More specifically, the loss value for the generator will change - the same generated image will receive different loss values as the discriminator improves.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-03T10:20:47.293Z
Comments.Text: @NeilSlater I think I came across in GANs only.
Comments.UserDisplayName: hanugm
Tags: <machine-learning><objective-functions><generative-adversarial-networks>
Title: When can we call a loss function "adaptive"?
UserDisplayName: hanugm
ThreadText: # When can we call a loss function "adaptive"?
  
  (Asked by: username_0 on 2021-09-03)
  
  A loss function is a measure of how bad our neural network is. We can decrease the loss by proper training.
  
  I came across the phrase "adaptive loss function" in several research papers. For example: consider the following excerpt from the "Introduction" of the research paper titled [Generative Adversarial Text to Image Synthesis](https://proceedings.mlr.press/v48/reed16.pdf) by Scott Reed et al.
  
  > By conditioning both generator and discriminator on side information, we can naturally model this phenomenon since the discriminator network acts as a "smart" **adaptive** loss function.
  
  When can we denote a loss function as adaptive? Is it a mathematical property or is solely based on the context?
  
  **Tags:** machine-learning, objective-functions, generative-adversarial-networks
  
  > **Comments:**
  > **username_1**: It is the normal English meaning of "adaptive".  
  > **username_0**: @user253751 So, if the loss function is changing according to the need, then can I say it as adaptive?  
  > **username_1**: Sure. It is the normal English meaning. It is not something with a precise mathematical definition, as far as I'm aware. AI papers are not a programming language.  
  > **username_2**: Did you come across the term in context other than GANs? I can see how the discriminator might be considered to present an "adaptive" loss function for GANs as user253751 states. More specifically, the loss value for the generator will change - the same generated image will receive different loss values as the discriminator improves.  
  > **username_0**: @username_2 I think I came across in GANs only.  
  
  ---
  
  ## 0 answers
  
  