Answers.Body: <p>The provably more-reliable approach to learning with noisy labels/label errors (instead of altering the model architecture or loss function) is to find the label errors and train without them on cleaned data -- a result shown in the <a href="https://cleanlab.ai/studio" rel="nofollow noreferrer">confident learning paper (Northcutt, Lu, Chuang; 2021)</a>). I am an author on this paper.</p>
  <p>Re-phrasing your question to two questions: (1) &quot;how can I generally find label errors in any dataset for any model?&quot; and (2) &quot;how can I train a model on noisy data as if it was clean data?&quot;.</p>
  <p>There are one line of code answers to each of these questions using the <a href="https://docs.cleanlab.ai" rel="nofollow noreferrer">cleanlab</a> python package which implements the algorithms and theory in the confident learning paper and works for <strong>any dataset you can train a classifier on</strong> for most data formats, ML and deep learning frameworks, and data modalities, e.g. image, text, tabular, and audio data.</p>
  <p>link to python package: <a href="https://github.com/cleanlab/cleanlab" rel="nofollow noreferrer">https://github.com/cleanlab/cleanlab</a></p>
  <h2>Find label issues in 1 line of code</h2>
  <pre><code>from cleanlab.classification import CleanLearning
  from cleanlab.filter import find_label_issues
  
  # Option 1 - works with sklearn-compatible models - just input the data and labels ツ
  label_issues_info = CleanLearning(clf=sklearn_compatible_model).find_label_issues(data, labels)
  
  # Option 2 - works with ANY ML model - just input the model's predicted probabilities
  ordered_label_issues = find_label_issues(
      labels=labels,
      pred_probs=pred_probs,  # out-of-sample predicted probabilities from any model
      return_indices_ranked_by='self_confidence',
  )
  </code></pre>
  <h2>Train a model as if the dataset did not have errors -- 3 lines of code</h2>
  <pre><code>from sklearn.linear_model import LogisticRegression
  from cleanlab.classification import CleanLearning
  
  cl = CleanLearning(clf=LogisticRegression())  # any sklearn-compatible classifier
  cl.fit(train_data, labels)
  
  # Estimate the predictions you would have gotten if you trained without mislabeled data.
  predictions = cl.predict(test_data)
  </code></pre>
  <p>Documentation and runnable tutorials for cleanlab: <a href="https://docs.cleanlab.ai/" rel="nofollow noreferrer">https://docs.cleanlab.ai/</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-04T20:37:58.633Z
Answers.LastActivityDate: 2022-11-04T20:37:58.633Z
Answers.UserDisplayName: cgnorthcutt
Body: <p>Let's assume I want to build a semantic segmentation algorithm, based on Multires-UNET. My GT-masks are messy and generated by a GAN, but they are getting better and better over time. The goal is knowledge expansion (based on the paper <a href="https://arxiv.org/abs/1911.04252" rel="nofollow noreferrer">Noisy-Student</a>). </p>
  
  <p>Can you generally say that PreLU and Leaky Relu are better for noisy labels (or imperfect ones), like the situation in GANs in general?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-30T20:14:23.400Z
Comments.Text: To my experience ReLU is superior to LeakyReLU.
Comments.UserDisplayName: CoffeDeveloper
Tags: <machine-learning><activation-functions><relu>
Title: Are PreLU and Leaky ReLU better than ReLU in the case of noisy labels?
UserDisplayName: Paul Higazi
ThreadText: # Are PreLU and Leaky ReLU better than ReLU in the case of noisy labels?
  
  (Asked by: username_0 on 2020-03-29)
  
  Let's assume I want to build a semantic segmentation algorithm, based on Multires-UNET. My GT-masks are messy and generated by a GAN, but they are getting better and better over time. The goal is knowledge expansion (based on the paper [Noisy-Student](https://arxiv.org/abs/1911.04252)).
  
  Can you generally say that PreLU and Leaky Relu are better for noisy labels (or imperfect ones), like the situation in GANs in general?
  
  **Tags:** machine-learning, activation-functions, relu
  > **Comments:**
  > **username_1**: To my experience ReLU is superior to LeakyReLU.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-11-04. Score: 0
  
  The provably more-reliable approach to learning with noisy labels/label errors (instead of altering the model architecture or loss function) is to find the label errors and train without them on cleaned data -- a result shown in the confident learning paper (Northcutt, Lu, Chuang; 2021)). I am an author on this paper.
  Re-phrasing your question to two questions: (1) "how can I generally find label errors in any dataset for any model?" and (2) "how can I train a model on noisy data as if it was clean data?".
  There are one line of code answers to each of these questions using the cleanlab python package which implements the algorithms and theory in the confident learning paper and works for any dataset you can train a classifier on for most data formats, ML and deep learning frameworks, and data modalities, e.g. image, text, tabular, and audio data.
  link to python package: https://github.com/cleanlab/cleanlab
  Find label issues in 1 line of code
  from cleanlab.classification import CleanLearning
  from cleanlab.filter import find_label_issues
  
  # Option 1 - works with sklearn-compatible models - just input the data and labels ツ
  label_issues_info = CleanLearning(clf=sklearn_compatible_model).find_label_issues(data, labels)
  
  # Option 2 - works with ANY ML model - just input the model's predicted probabilities
  ordered_label_issues = find_label_issues(
      labels=labels,
      pred_probs=pred_probs,  # out-of-sample predicted probabilities from any model
      return_indices_ranked_by='self_confidence',
  )
  
  Train a model as if the dataset did not have errors -- 3 lines of code
  from sklearn.linear_model import LogisticRegression
  from cleanlab.classification import CleanLearning
  
  cl = CleanLearning(clf=LogisticRegression())  # any sklearn-compatible classifier
  cl.fit(train_data, labels)
  
  # Estimate the predictions you would have gotten if you trained without mislabeled data.
  predictions = cl.predict(test_data)
  
  Documentation and runnable tutorials for cleanlab: https://docs.cleanlab.ai/
  
  
  ---
  
  