Answers.Body: <p>Yes, you can weight the loss function for each example, so that instead of your cost function being</p>
  
  <p><span class="math-container">$$J = \sum_i \mathcal{L}(y_i, \hat{y}_i)$$</span></p>
  
  <p>It will be </p>
  
  <p><span class="math-container">$$J = \sum_i w_i\mathcal{L}(y_i, \hat{y}_i)$$</span></p>
  
  <p>Where <span class="math-container">$i$</span> iterates over your data set, <span class="math-container">$\mathcal{L}$</span> is the loss function you are using, <span class="math-container">$y_i$</span> is ground truth for each example and <span class="math-container">$\hat{y}_i$</span> is prediction for each example.</p>
  
  <p>You can set your relevance weighting according to any criteria you like based on the example, and your dataset/goals. So for instance you could set it to <span class="math-container">$1.0$</span> for complete examples and <span class="math-container">$0.1$</span> for incomplete examples. Depending on your NN framework, it may already offer per-example weighting, and even if it does not, auto-differentiation means that typically you only need to implement the forward logic into the cost function for each minibatch, and the weighting will be applied correctly to gradients with no more work required. If you do need to implement gradient calculations yourself, you simply multiply the initial gradient of each example by the example's weight <span class="math-container">$w_i$</span>.</p>
  
  <p>Once you make a change like this, you need to take care on how you set up and interpret your test set. When your model is used in production, if there are still inputs with missing details, you probably don't want to weight the metrics (e.g. accuracy rating) in the same way, but to report it as-is against a correctly sampled dataset of unseen examples from the population of expected inputs.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-06T10:00:19.950Z
Answers.LastActivityDate: 2020-03-06T10:12:58.267Z
Answers.LastEditDate: 2020-03-06T10:12:58.267Z
Answers.UserDisplayName: Neil Slater
Body: <p>I have a lot of empty values in my dataset, so I want to let my neural network 'learn more' on the rows that have no empty values because these rows are of higher importance.</p>
  
  <p>Is there a way to do this?</p>
  
Tags: <neural-networks><datasets>
Title: Can you let specific data impact a neural network more than other data?
UserDisplayName: Yari Nowicki
ThreadText: # Can you let specific data impact a neural network more than other data?
  
  (Asked by: username_0 on 2020-03-06)
  
  I have a lot of empty values in my dataset, so I want to let my neural network 'learn more' on the rows that have no empty values because these rows are of higher importance.
  
  Is there a way to do this?
  
  **Tags:** neural-networks, datasets
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-03-06. Score: 3 | Accepted answer
  
  Yes, you can weight the loss function for each example, so that instead of your cost function being
  
  $$J = \sum\_i \mathcal{L}(y\_i, \hat{y}\_i)$$
  
  It will be
  
  $$J = \sum\_i w\_i\mathcal{L}(y\_i, \hat{y}\_i)$$
  
  Where $i$ iterates over your data set, $\mathcal{L}$ is the loss function you are using, $y\_i$ is ground truth for each example and $\hat{y}\_i$ is prediction for each example.
  
  You can set your relevance weighting according to any criteria you like based on the example, and your dataset/goals. So for instance you could set it to $1.0$ for complete examples and $0.1$ for incomplete examples. Depending on your NN framework, it may already offer per-example weighting, and even if it does not, auto-differentiation means that typically you only need to implement the forward logic into the cost function for each minibatch, and the weighting will be applied correctly to gradients with no more work required. If you do need to implement gradient calculations yourself, you simply multiply the initial gradient of each example by the example's weight $w\_i$.
  
  Once you make a change like this, you need to take care on how you set up and interpret your test set. When your model is used in production, if there are still inputs with missing details, you probably don't want to weight the metrics (e.g. accuracy rating) in the same way, but to report it as-is against a correctly sampled dataset of unseen examples from the population of expected inputs.
  
  
  ---
  
  