Answers.Body: <p>It all depends on the quality of data. Due to old rule "Garbage in, garbage out" <a href="https://en.wikipedia.org/wiki/Garbage_in,_garbage_out" rel="nofollow noreferrer">link</a>
  , if you have bad quality data(data redundancy, unstructured data, too much memory, etc) your results won't be spectacular. </p>
  
  <p>In other cases, everybody could be a Data Scientist, because its only task was "put raw text into classifier". Also, you should remember that BERT or GPT-2 it's deep learning algorithms so they not need too much processing. Using preprocessing in machine learning is more that needed(prediction of sentiment for example).</p>
  
  <p>Shortly, preprocessing is <strong>optional</strong>, but highly <strong>advisable</strong>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-12T01:00:48.573Z
Answers.Text: Depends, I agree in some tasks it can help. But I don't think in general we should think about raw data as "garbage' necessarily.
Answers.UserDisplayName: information_interchange
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-28T20:58:44.763Z
Answers.LastActivityDate: 2020-04-26T10:00:58.640Z
Answers.LastEditDate: 2020-04-26T10:00:58.640Z
Answers.UserDisplayName: fuwiak
Answers.Body: <p>It depends on the dataset we have and algorithm we use, usually text preprocessing can help your model perform better. But, some preprocessing method can have no significant impact on accuracy. We need to choose which preprocessing method that can help us make better quality dataset to give to the model. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-21T10:45:44.787Z
Answers.LastActivityDate: 2020-04-21T10:45:44.787Z
Answers.UserDisplayName: Beladina Elfitri
Body: <p>As a first step in many NLP courses, we learn about text preprocessing. The steps include lemmatization, removal of rare words, correcting typos etc. But I am not so sure about the actual effectiveness of doing such a step; in particular, if we are learning a neural network for a downstream task, it seems like modern state of the art (BERT, GPT-2) just take essentially raw input. </p>
  
  <p>For instance, <a href="https://www.aclweb.org/anthology/W18-5406.pdf" rel="nofollow noreferrer">this ACL paper</a> seems to show that the result of text preprocessing is mixed, to say the least. </p>
  
  <p>So is text preprocessing really all that necessary for NLP? In particular, I want to contrast/compare against vision and tabular data, where I have empirically found that standardization usually actually does help. Feel free to share your personal experiences/what use cases where text preprocessing helps! </p>
  
Tags: <neural-networks><natural-language-processing><data-preprocessing><structured-data>
Title: Is text preprocessing really all that necessary for NLP?
UserDisplayName: information_interchange
ThreadText: # Is text preprocessing really all that necessary for NLP?
  
  (Asked by: username_0 on 2020-03-28)
  
  As a first step in many NLP courses, we learn about text preprocessing. The steps include lemmatization, removal of rare words, correcting typos etc. But I am not so sure about the actual effectiveness of doing such a step; in particular, if we are learning a neural network for a downstream task, it seems like modern state of the art (BERT, GPT-2) just take essentially raw input.
  
  For instance, [this ACL paper](https://www.aclweb.org/anthology/W18-5406.pdf) seems to show that the result of text preprocessing is mixed, to say the least.
  
  So is text preprocessing really all that necessary for NLP? In particular, I want to contrast/compare against vision and tabular data, where I have empirically found that standardization usually actually does help. Feel free to share your personal experiences/what use cases where text preprocessing helps!
  
  **Tags:** neural-networks, natural-language-processing, data-preprocessing, structured-data
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2020-03-28. Score: 2
  
  It all depends on the quality of data. Due to old rule "Garbage in, garbage out" [link](https://en.wikipedia.org/wiki/Garbage_in,_garbage_out)
  , if you have bad quality data(data redundancy, unstructured data, too much memory, etc) your results won't be spectacular.
  
  In other cases, everybody could be a Data Scientist, because its only task was "put raw text into classifier". Also, you should remember that BERT or GPT-2 it's deep learning algorithms so they not need too much processing. Using preprocessing in machine learning is more that needed(prediction of sentiment for example).
  
  Shortly, preprocessing is **optional**, but highly **advisable**.
  
  > **Comments:**
  > **username_0**: Depends, I agree in some tasks it can help. But I don't think in general we should think about raw data as "garbage' necessarily.  
  
  ---
  
  ### Answer by username_2 on 2020-04-21. Score: 0
  
  It depends on the dataset we have and algorithm we use, usually text preprocessing can help your model perform better. But, some preprocessing method can have no significant impact on accuracy. We need to choose which preprocessing method that can help us make better quality dataset to give to the model.
  
  
  ---
  
  