<p>This is referred to as <em><strong>label noise</strong></em> (or <em>noisy labels</em>), and it is indeed different from just &quot;noisy data&quot;; quoting from the 2014 paper <a href="https://www.researchgate.net/publication/261601383_Classification_in_the_Presence_of_Label_Noise_A_Survey" rel="nofollow noreferrer">Classiﬁcation in the Presence of Label Noise: a Survey</a>:</p>
<blockquote>
<p>In this survey,
it is assumed that each training sample is associated with an
observed label. This label often corresponds to the true class of
the sample, but it may be <em>subjected to a noise process before
being presented to the learning algorithm</em>. It is therefore
important to distinguish the true class of an instance from its
observed label. The process which pollutes labels is called
label noise and must be separated from feature (or attribute)
noise which affects the value of features.</p>
</blockquote>
<p>You may also find the (preprint) paper <a href="https://arxiv.org/abs/2003.10471" rel="nofollow noreferrer">Label Noise Types and Their Effects on Deep Learning</a> useful; there, the authors propose a taxonomy of different types of label noise:</p>
<blockquote>
<p>Label noise types can be subdivided into
three main groups as follows.</p>
<p>• <strong>Uniform noise</strong>: Flipping probability of label from its
true class to any other class is equally distributed. Many
works in literature use synthetic uniform label noise by
just flipping labels randomly for a given percentage of
data instances [21]–[24].</p>
<p>• <strong>Class-dependent noise</strong>: Flipping probability of label
depends on the true class of the data instance. This
is mostly represented by a confusion matrix and can
be designed in different ways. The easiest way is to
attain inter-class transition probabilities just random [25],
so that there is still class dependence since transition
probabilities are given according to classes but without
any correlation to class similarities. In a more structured
way, noise transition matrix can be designed in a way that
similar classes have a bigger probability to be flipped to
each other [26]–[30]. Some works use pairwise noise,
in which transition from one class can only be defined
to one another class [31]–[35]. Work of [36] checks the
popularity of classes and constructs transition matrix so
that mislabeling happens from popular class to unpopular
class or vice versa.</p>
<p>• <strong>Feature-dependent noise</strong>: The probability of mislabeling
depends on features of instances. In order to generate
feature-dependent noise, features of each instance should
be extracted, and their similarities to other instances from
different classes should be evaluated. Unlike uniform and
class-dependent noise, there are much fewer implemen-
tations of synthetic feature-dependent label noise. One
particular work in this field is [37], where data is clustered
with the kNN algorithm, and labels are flipped randomly
for clusters of data. This method provides concentrated
noise in the feature space. But, this type of synthetic noise
doesn’t evaluate the instance similarities and therefore
different from our proposed approach. Alternatively, in
case there is a surrounding text for each image in the
dataset, some works create noisy labels from the interpretations of these texts [38]–[41], assuming surrounding
texts are related to features of data. But this approach is
restricted to datasets with surrounding user-defined texts,
which is not the case for most of the time.</p>
</blockquote>
<p>Other possibly useful resources on the subject:</p>
<ul>
<li><a href="https://www.cs.bham.ac.uk/%7Eaxk/ESANNtut.pdf" rel="nofollow noreferrer">A Comprehensive Introduction to Label Noise</a> (conference paper, 2014)</li>
<li><a href="http://knowdive.disi.unitn.it/2018/09/a-little-talk-on-label-noise/" rel="nofollow noreferrer">A Little Talk on Label Noise</a> (blog post, 2018)</li>
<li><a href="https://towardsdatascience.com/data-noise-and-label-noise-in-machine-learning-98c8a3c8322e" rel="nofollow noreferrer">Data Noise and Label Noise in Machine Learning</a> (blog post, 2021)</li>
</ul>

CC BY-SA 4.0
2022-01-30T14:48:23.113Z
2022-01-30T14:59:11.173Z
2022-01-30T14:59:11.173Z
desertnaut
<p>I wonder if following characteristic of data has some standard &quot;professional&quot; or scientific term associated with it.</p>
<p>Let's assume that I have a set of dog/cat images labeled 0 for a cat and 1 for a dog. My purpose is to extract (classify) as many true dog images as possible, with the smallest amount of non-dog images. But the problem with data is that:</p>
<ul>
<li>some dogs images are labeled as 0</li>
<li>some cats images are labeled as 1</li>
<li>some images are of other animals or non-animal objects and can have both labels 1 and 0</li>
</ul>
<p>Is there any specific term describing such cases, or they are  just &quot;noisy&quot; data?</p>

<classification><terminology><datasets><data-labelling><training-datasets>
Is there a standard term for the following flaw in the data?
GKozinski
# Is there a standard term for the following flaw in the data?

(Asked by: username_0 on 2022-01-29)

I wonder if following characteristic of data has some standard "professional" or scientific term associated with it.

Let's assume that I have a set of dog/cat images labeled 0 for a cat and 1 for a dog. My purpose is to extract (classify) as many true dog images as possible, with the smallest amount of non-dog images. But the problem with data is that:

* some dogs images are labeled as 0
* some cats images are labeled as 1
* some images are of other animals or non-animal objects and can have both labels 1 and 0

Is there any specific term describing such cases, or they are just "noisy" data?

**Tags:** classification, terminology, datasets, data-labelling, training-datasets


---

## 1 answers

### Answer by username_1 on 2022-01-30. Score: 0 | Accepted answer

This is referred to as ***label noise*** (or *noisy labels*), and it is indeed different from just "noisy data"; quoting from the 2014 paper [Classiﬁcation in the Presence of Label Noise: a Survey](https://www.researchgate.net/publication/261601383_Classification_in_the_Presence_of_Label_Noise_A_Survey):

> In this survey,
> it is assumed that each training sample is associated with an
> observed label. This label often corresponds to the true class of
> the sample, but it may be *subjected to a noise process before
> being presented to the learning algorithm*. It is therefore
> important to distinguish the true class of an instance from its
> observed label. The process which pollutes labels is called
> label noise and must be separated from feature (or attribute)
> noise which affects the value of features.

You may also find the (preprint) paper [Label Noise Types and Their Effects on Deep Learning](https://arxiv.org/abs/2003.10471) useful; there, the authors propose a taxonomy of different types of label noise:

> Label noise types can be subdivided into
> three main groups as follows.
>
> • **Uniform noise**: Flipping probability of label from its
> true class to any other class is equally distributed. Many
> works in literature use synthetic uniform label noise by
> just flipping labels randomly for a given percentage of
> data instances [21]–[24].
>
> • **Class-dependent noise**: Flipping probability of label
> depends on the true class of the data instance. This
> is mostly represented by a confusion matrix and can
> be designed in different ways. The easiest way is to
> attain inter-class transition probabilities just random [25],
> so that there is still class dependence since transition
> probabilities are given according to classes but without
> any correlation to class similarities. In a more structured
> way, noise transition matrix can be designed in a way that
> similar classes have a bigger probability to be flipped to
> each other [26]–[30]. Some works use pairwise noise,
> in which transition from one class can only be defined
> to one another class [31]–[35]. Work of [36] checks the
> popularity of classes and constructs transition matrix so
> that mislabeling happens from popular class to unpopular
> class or vice versa.
>
> • **Feature-dependent noise**: The probability of mislabeling
> depends on features of instances. In order to generate
> feature-dependent noise, features of each instance should
> be extracted, and their similarities to other instances from
> different classes should be evaluated. Unlike uniform and
> class-dependent noise, there are much fewer implemen-
> tations of synthetic feature-dependent label noise. One
> particular work in this field is [37], where data is clustered
> with the kNN algorithm, and labels are flipped randomly
> for clusters of data. This method provides concentrated
> noise in the feature space. But, this type of synthetic noise
> doesn’t evaluate the instance similarities and therefore
> different from our proposed approach. Alternatively, in
> case there is a surrounding text for each image in the
> dataset, some works create noisy labels from the interpretations of these texts [38]–[41], assuming surrounding
> texts are related to features of data. But this approach is
> restricted to datasets with surrounding user-defined texts,
> which is not the case for most of the time.

Other possibly useful resources on the subject:

* [A Comprehensive Introduction to Label Noise](https://www.cs.bham.ac.uk/%7Eaxk/ESANNtut.pdf) (conference paper, 2014)
* [A Little Talk on Label Noise](http://knowdive.disi.unitn.it/2018/09/a-little-talk-on-label-noise/) (blog post, 2018)
* [Data Noise and Label Noise in Machine Learning](https://towardsdatascience.com/data-noise-and-label-noise-in-machine-learning-98c8a3c8322e) (blog post, 2021)


---

