Answers.Body: <p>The provably more-reliable approach to learning with noisy labels/label errors (instead of altering the model architecture or loss function) is to find the label errors and train without them on cleaned data -- a result shown in the <a href="https://cleanlab.ai/studio" rel="nofollow noreferrer">confident learning paper (Northcutt, Lu, Chuang; 2021)</a>). I am an author on this paper.</p>
  <p>Re-phrasing your question to two questions: (1) &quot;how can I generally find label errors in any dataset for any model?&quot; and (2) &quot;how can I train a model on noisy data as if it was clean data?&quot;.</p>
  <p>There are one line of code answers to each of these questions using the <a href="https://docs.cleanlab.ai" rel="nofollow noreferrer">cleanlab</a> python package which implements the algorithms and theory in the confident learning paper and works for <strong>any dataset you can train a classifier on</strong> for most data formats, ML and deep learning frameworks, and data modalities, e.g. image, text, tabular, and audio data.</p>
  <p>link to python package: <a href="https://github.com/cleanlab/cleanlab" rel="nofollow noreferrer">https://github.com/cleanlab/cleanlab</a></p>
  <h2>Find label issues in 1 line of code</h2>
  <pre><code>from cleanlab.classification import CleanLearning
  from cleanlab.filter import find_label_issues
  
  # Option 1 - works with sklearn-compatible models - just input the data and labels ツ
  label_issues_info = CleanLearning(clf=sklearn_compatible_model).find_label_issues(data, labels)
  
  # Option 2 - works with ANY ML model - just input the model's predicted probabilities
  ordered_label_issues = find_label_issues(
      labels=labels,
      pred_probs=pred_probs,  # out-of-sample predicted probabilities from any model
      return_indices_ranked_by='self_confidence',
  )
  </code></pre>
  <h2>Train a model as if the dataset did not have errors -- 3 lines of code</h2>
  <pre><code>from sklearn.linear_model import LogisticRegression
  from cleanlab.classification import CleanLearning
  
  cl = CleanLearning(clf=LogisticRegression())  # any sklearn-compatible classifier
  cl.fit(train_data, labels)
  
  # Estimate the predictions you would have gotten if you trained without mislabeled data.
  predictions = cl.predict(test_data)
  </code></pre>
  <p>Documentation and runnable tutorials for cleanlab: <a href="https://docs.cleanlab.ai/" rel="nofollow noreferrer">https://docs.cleanlab.ai/</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-04T20:37:58.633Z
Answers.LastActivityDate: 2022-11-04T20:37:58.633Z
Answers.UserDisplayName: cgnorthcutt
Body: <p>Let's assume I want to build a semantic segmentation algorithm, based on Multires-UNET. My GT-masks are messy and generated by a GAN, but they are getting better and better over time. The goal is knowledge expansion (based on the paper <a href="https://arxiv.org/abs/1911.04252" rel="nofollow noreferrer">Noisy-Student</a>). </p>
  
  <p>Can you generally say that PreLU and Leaky Relu are better for noisy labels (or imperfect ones), like the situation in GANs in general?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-30T20:14:23.400Z
Comments.Text: To my experience ReLU is superior to LeakyReLU.
Comments.UserDisplayName: CoffeDeveloper
Tags: <machine-learning><activation-functions><relu>
Title: Are PreLU and Leaky ReLU better than ReLU in the case of noisy labels?
UserDisplayName: Paul Higazi
ThreadText: # Are PreLU and Leaky ReLU better than ReLU in the case of noisy labels?
  
  (Asked by: username_0 on 2020-03-29)
  
  Let's assume I want to build a semantic segmentation algorithm, based on Multires-UNET. My GT-masks are messy and generated by a GAN, but they are getting better and better over time. The goal is knowledge expansion (based on the paper [Noisy-Student](https://arxiv.org/abs/1911.04252)).
  
  Can you generally say that PreLU and Leaky Relu are better for noisy labels (or imperfect ones), like the situation in GANs in general?
  
  **Tags:** machine-learning, activation-functions, relu
  
  > **Comments:**
  > **username_1**: To my experience ReLU is superior to LeakyReLU.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-11-04. Score: 0
  
  The provably more-reliable approach to learning with noisy labels/label errors (instead of altering the model architecture or loss function) is to find the label errors and train without them on cleaned data -- a result shown in the [confident learning paper (Northcutt, Lu, Chuang; 2021)](https://cleanlab.ai/studio)). I am an author on this paper.
  
  Re-phrasing your question to two questions: (1) "how can I generally find label errors in any dataset for any model?" and (2) "how can I train a model on noisy data as if it was clean data?".
  
  There are one line of code answers to each of these questions using the [cleanlab](https://docs.cleanlab.ai) python package which implements the algorithms and theory in the confident learning paper and works for **any dataset you can train a classifier on** for most data formats, ML and deep learning frameworks, and data modalities, e.g. image, text, tabular, and audio data.
  
  link to python package: <https://github.com/cleanlab/cleanlab>
  
  ## Find label issues in 1 line of code
  
  ```
  from cleanlab.classification import CleanLearning
  from cleanlab.filter import find_label_issues
  
  # Option 1 - works with sklearn-compatible models - just input the data and labels ツ
  label_issues_info = CleanLearning(clf=sklearn_compatible_model).find_label_issues(data, labels)
  
  # Option 2 - works with ANY ML model - just input the model's predicted probabilities
  ordered_label_issues = find_label_issues(
      labels=labels,
      pred_probs=pred_probs,  # out-of-sample predicted probabilities from any model
      return_indices_ranked_by='self_confidence',
  )
  
  ```
  
  ## Train a model as if the dataset did not have errors -- 3 lines of code
  
  ```
  from sklearn.linear_model import LogisticRegression
  from cleanlab.classification import CleanLearning
  
  cl = CleanLearning(clf=LogisticRegression())  # any sklearn-compatible classifier
  cl.fit(train_data, labels)
  
  # Estimate the predictions you would have gotten if you trained without mislabeled data.
  predictions = cl.predict(test_data)
  
  ```
  
  Documentation and runnable tutorials for cleanlab: <https://docs.cleanlab.ai/>
  
  
  ---
  
  