Answers.Body: <p>The most general solution today for the problem of finding label errors in datasets is called &quot;confident learning&quot; which works for all datasets and models, can be run time-efficiently <a href="https://github.com/cleanlab/cleanlab" rel="nofollow noreferrer">in one line of code using cleanlab</a>, and has substantial theory to prove that it works in realistic conditions on real-world datasets. This &quot;confident learning&quot; paper was a culminating result during my PhD at MIT and I am an author on the paper.</p>
  <ul>
  <li>The Confident Learning Paper: <a href="https://arxiv.org/abs/1911.00068" rel="nofollow noreferrer">https://arxiv.org/abs/1911.00068</a></li>
  <li>Code: <a href="https://github.com/cleanlab/cleanlab" rel="nofollow noreferrer">https://github.com/cleanlab/cleanlab</a></li>
  <li>Examples of label errors found with cleanlab: <a href="https://labelerrors.com" rel="nofollow noreferrer">https://labelerrors.com</a></li>
  </ul>
  <h2>Find label issues in your dataset in 1 line of code</h2>
  <pre><code>from cleanlab.classification import CleanLearning
  from cleanlab.filter import find_label_issues
  
  # Option 1 - works with sklearn-compatible models - just input the data and labels ツ
  label_issues_info = CleanLearning(clf=sklearn_compatible_model).find_label_issues(data, labels)
  
  # Option 2 - works with ANY ML model - just input the model's predicted probabilities
  ordered_label_issues = find_label_issues(
      labels=labels,
      pred_probs=pred_probs,  # out-of-sample predicted probabilities from any model
      return_indices_ranked_by='self_confidence',
  )
  </code></pre>
  <h2>Train a model as if the dataset did not have errors -- 3 lines of code</h2>
  <pre><code>from sklearn.linear_model import LogisticRegression
  from cleanlab.classification import CleanLearning
  
  cl = CleanLearning(clf=LogisticRegression())  # any sklearn-compatible classifier
  cl.fit(train_data, labels)
  
  # Estimate the predictions you would have gotten if you trained without mislabeled data.
  predictions = cl.predict(test_data)
  </code></pre>
  <p>Documentation and runnable tutorials for cleanlab: <a href="https://docs.cleanlab.ai/" rel="nofollow noreferrer">https://docs.cleanlab.ai/</a></p>
  <h2>The above approaches find issues. To correct label issues in your dataset:</h2>
  <p>The above approaches, just find issues and train without them. To correct the labels or train a model on a corrected dataset (that still includes all the errors, but now corrected with the right label), there is a no-code tool for that called <a href="https://cleanlab.ai/studio" rel="nofollow noreferrer">Cleanlab Studio</a> (<a href="https://cleanlab.ai/studio" rel="nofollow noreferrer">https://cleanlab.ai/studio</a>) for which I am also an author.</p>
  <h2>Background if you're interested:</h2>
  <p>I spent half a decade working with <a href="https://en.wikipedia.org/wiki/Isaac_Chuang" rel="nofollow noreferrer">Isaac Chuang</a> (inventor of the quantum computer) to solve this problem in a way that works for every dataset and every model (and every future dataset and future model) during my PhD at MIT. I originally decided to solve this problem after discovering (while building MIT and Harvard's cheating detection system in 2013) that most real-world datasets have significant label errors and this is one of the biggest problems that companies and universities struggle with when training ML models.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-03T22:53:54.480Z
Answers.LastActivityDate: 2022-11-03T22:53:54.480Z
Answers.UserDisplayName: cgnorthcutt
Body: <p>I am doing literature research on algorithms for correcting mislabeled data using multilayer perceptrons. Found an "old" paper <a href="https://pdfs.semanticscholar.org/3a09/c6f3873dd0327a3f9d83b7ff1c8b43a06cc3.pdf" rel="nofollow noreferrer">An algorithm for correcting mislabeled data</a> (2001) by Xinchuan Zeng et al. Please share if you are aware of recent/current updates with a brief thoughts. Thanks in advance.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-17T15:28:25.743Z
Comments.Text: Are you just looking for techniques that specifically use MLPs or any other technique for correcting mislabelled data?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-17T15:53:27.577Z
Comments.Text: I have numerical data with "unrelated" columns of numbers. Therefore, I think MLPs make more sense rather than, for example, CNNs.
Comments.UserDisplayName: ViB
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-17T15:56:04.300Z
Comments.Text: I am also not sure if https://arxiv.org/abs/1801.08019 and https://ieeexplore.ieee.org/abstract/document/6413805 will be more powerful as compared to MLPs.
Comments.UserDisplayName: ViB
Tags: <neural-networks><deep-learning><papers><multilayer-perceptrons><multi-label-classification>
Title: Recent algorithms for correcting mislabeled data using multilayer perceptrons
UserDisplayName: ViB
ThreadText: # Recent algorithms for correcting mislabeled data using multilayer perceptrons
  
  (Asked by: username_0 on 2020-03-16)
  
  I am doing literature research on algorithms for correcting mislabeled data using multilayer perceptrons. Found an "old" paper [An algorithm for correcting mislabeled data](https://pdfs.semanticscholar.org/3a09/c6f3873dd0327a3f9d83b7ff1c8b43a06cc3.pdf) (2001) by Xinchuan Zeng et al. Please share if you are aware of recent/current updates with a brief thoughts. Thanks in advance.
  
  **Tags:** neural-networks, deep-learning, papers, multilayer-perceptrons, multi-label-classification
  
  > **Comments:**
  > **username_1**: Are you just looking for techniques that specifically use MLPs or any other technique for correcting mislabelled data?  
  > **username_0**: I have numerical data with "unrelated" columns of numbers. Therefore, I think MLPs make more sense rather than, for example, CNNs.  
  > **username_0**: I am also not sure if https://arxiv.org/abs/1801.08019 and https://ieeexplore.ieee.org/abstract/document/6413805 will be more powerful as compared to MLPs.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-11-03. Score: 1
  
  The most general solution today for the problem of finding label errors in datasets is called "confident learning" which works for all datasets and models, can be run time-efficiently [in one line of code using cleanlab](https://github.com/cleanlab/cleanlab), and has substantial theory to prove that it works in realistic conditions on real-world datasets. This "confident learning" paper was a culminating result during my PhD at MIT and I am an author on the paper.
  
  * The Confident Learning Paper: <https://arxiv.org/abs/1911.00068>
  * Code: <https://github.com/cleanlab/cleanlab>
  * Examples of label errors found with cleanlab: <https://labelerrors.com>
  
  ## Find label issues in your dataset in 1 line of code
  
  ```
  from cleanlab.classification import CleanLearning
  from cleanlab.filter import find_label_issues
  
  # Option 1 - works with sklearn-compatible models - just input the data and labels ツ
  label_issues_info = CleanLearning(clf=sklearn_compatible_model).find_label_issues(data, labels)
  
  # Option 2 - works with ANY ML model - just input the model's predicted probabilities
  ordered_label_issues = find_label_issues(
      labels=labels,
      pred_probs=pred_probs,  # out-of-sample predicted probabilities from any model
      return_indices_ranked_by='self_confidence',
  )
  
  ```
  
  ## Train a model as if the dataset did not have errors -- 3 lines of code
  
  ```
  from sklearn.linear_model import LogisticRegression
  from cleanlab.classification import CleanLearning
  
  cl = CleanLearning(clf=LogisticRegression())  # any sklearn-compatible classifier
  cl.fit(train_data, labels)
  
  # Estimate the predictions you would have gotten if you trained without mislabeled data.
  predictions = cl.predict(test_data)
  
  ```
  
  Documentation and runnable tutorials for cleanlab: <https://docs.cleanlab.ai/>
  
  ## The above approaches find issues. To correct label issues in your dataset:
  
  The above approaches, just find issues and train without them. To correct the labels or train a model on a corrected dataset (that still includes all the errors, but now corrected with the right label), there is a no-code tool for that called [Cleanlab Studio](https://cleanlab.ai/studio) (<https://cleanlab.ai/studio>) for which I am also an author.
  
  ## Background if you're interested:
  
  I spent half a decade working with [Isaac Chuang](https://en.wikipedia.org/wiki/Isaac_Chuang) (inventor of the quantum computer) to solve this problem in a way that works for every dataset and every model (and every future dataset and future model) during my PhD at MIT. I originally decided to solve this problem after discovering (while building MIT and Harvard's cheating detection system in 2013) that most real-world datasets have significant label errors and this is one of the biggest problems that companies and universities struggle with when training ML models.
  
  
  ---
  
  