<p>This comes from the fact that you want the same magnitude from the loss. Think of it this way: a non-weighted loss function actually has all its weights to 1 and so over the whole data set, samples are weighted with 1 and the sum of all weights is therefore <span class="math-container">$N$</span>, if <span class="math-container">$N$</span> is the total number of samples.</p>

<p>Now in the case of a weighted loss, we want the weights to also sum to <span class="math-container">$N$</span> so that the loss's magnitude is comparable (<span class="math-container">$i = 1..C$</span> are your classes, <span class="math-container">$N_i$</span> is the number of samples for class <span class="math-container">$i$</span>):</p>

<p><span class="math-container">$$S = \sum_{i=0\ }^{C} \sum_{s_i=1}^{N_i} w_{i} = \sum_{i=0}^{i}\sum_{s_i=1}^{N_i}\frac{1}{N_i} \frac{N}{C} = \frac{N}{C} \sum_{i=0}^{C}\sum_{s_i=1}^{N_i}\frac{1}{N_i} = \frac{N}{C} \sum_{i=0}^{C}N_i\frac{1}{N_i}  = \frac{N}{C} \sum_{i=0}^{C}1 = \frac{N}{C} C = N$$</span></p>

CC BY-SA 4.0
2020-04-14T13:10:03.737Z
2020-04-14T13:10:03.737Z
ted
<p>I am trying to build a classifier which should be trained with the cross entropy loss. The training data is highly class-imbalanced. To tackle this, I've gone through the <a href="https://www.tensorflow.org/tutorials/structured_data/imbalanced_data#calculate_class_weights" rel="nofollow noreferrer">advice of the tensorflow docs</a></p>

<p>and now I am using a weighted cross entropy loss where the weights are calculated as </p>

<pre><code>weight_for_class_a = (1 / samples_for_class_a) * total_number_of_samples/number_of_classes
</code></pre>

<p>following the mentioned tutorial. </p>

<p>It works perfectly, but why is there this factor <code>total_number_of_samples/number_of_classes</code>?
The mentioned tutorial says this</p>

<blockquote>
  <p>[...] helps keep the loss to a similar magnitude.</p>
</blockquote>

<p>But I don not understand why. Can someone clarify?</p>

<machine-learning><classification><datasets><weights><cross-entropy>
How are weights for weighted x-entropy loss on imbalanced data calculated?
jmatin
# How are weights for weighted x-entropy loss on imbalanced data calculated?

(Asked by: username_0 on 2020-04-14)

I am trying to build a classifier which should be trained with the cross entropy loss. The training data is highly class-imbalanced. To tackle this, I've gone through the advice of the tensorflow docs
and now I am using a weighusername_1 cross entropy loss where the weights are calculausername_1 as 
weight_for_class_a = (1 / samples_for_class_a) * total_number_of_samples/number_of_classes

following the mentioned tutorial. 
It works perfectly, but why is there this factor total_number_of_samples/number_of_classes?
The mentioned tutorial says this

[...] helps keep the loss to a similar magnitude.

But I don not understand why. Can someone clarify?

**Tags:** machine-learning, classification, datasets, weights, cross-entropy


---

## 1 answers

### Answer by username_1 on 2020-04-14. Score: 0

This comes from the fact that you want the same magnitude from the loss. Think of it this way: a non-weighusername_1 loss function actually has all its weights to 1 and so over the whole data set, samples are weighusername_1 with 1 and the sum of all weights is therefore $N$, if $N$ is the total number of samples.

Now in the case of a weighusername_1 loss, we want the weights to also sum to $N$ so that the loss's magnitude is comparable ($i = 1..C$ are your classes, $N\_i$ is the number of samples for class $i$):

$$S = \sum\_{i=0\ }^{C} \sum\_{s\_i=1}^{N\_i} w\_{i} = \sum\_{i=0}^{i}\sum\_{s\_i=1}^{N\_i}\frac{1}{N\_i} \frac{N}{C} = \frac{N}{C} \sum\_{i=0}^{C}\sum\_{s\_i=1}^{N\_i}\frac{1}{N\_i} = \frac{N}{C} \sum\_{i=0}^{C}N\_i\frac{1}{N\_i} = \frac{N}{C} \sum\_{i=0}^{C}1 = \frac{N}{C} C = N$$


---

