Answers.Body: <p>I'm assuming you mean the original loss explained in <a href="https://arxiv.org/pdf/1406.2661.pdf" rel="nofollow noreferrer">the original GAN paper</a>. This practice was by design and for interpretation. An expectation is what your expecting to get, which is generally a good objective when your are sampling. Note this isn't just in GANs, it's in most objective functions used in a widespread of problems.</p>
  <p>In general practice, though, dividing is good because it works as a normalization. Let's say you train one batch with 10 elements and the next with 8. Wouldn't you want each training example to be weighted equally?</p>
  <p>Also, even if you use equal-sized batches, then the &quot;division&quot; you mention gets eaten up by the learning rate if you're using some form of gradient scheme.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-10T00:57:50.627Z
Answers.LastActivityDate: 2020-09-27T19:56:30.567Z
Answers.LastEditDate: 2020-09-27T19:56:30.567Z
Answers.UserDisplayName: mshlis
Body: <p>Why do the GAN's loss functions use an expectation (sum + division) instead of a simple sum?</p>
  
Tags: <neural-networks><machine-learning><generative-adversarial-networks>
Title: Why is an expectation used instead of simple sum in GANs?
UserDisplayName: NNLearner
ThreadText: # Why is an expectation used instead of simple sum in GANs?
  
  (Asked by: username_0 on 2019-06-09)
  
  Why do the GAN's loss functions use an expectation (sum + division) instead of a simple sum?
  
  **Tags:** neural-networks, machine-learning, generative-adversarial-networks
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-06-10. Score: 0 | Accepted answer
  
  I'm assuming you mean the original loss explained in [the original GAN paper](https://arxiv.org/pdf/1406.2661.pdf). This practice was by design and for interpretation. An expectation is what your expecting to get, which is generally a good objective when your are sampling. Note this isn't just in GANs, it's in most objective functions used in a widespread of problems.
  
  In general practice, though, dividing is good because it works as a normalization. Let's say you train one batch with 10 elements and the next with 8. Wouldn't you want each training example to be weighted equally?
  
  Also, even if you use equal-sized batches, then the "division" you mention gets eaten up by the learning rate if you're using some form of gradient scheme.
  
  
  ---
  
  