Body: <p>Regarding the use of pre-processing techniques before using Transformers models, I read <a href="https://stackoverflow.com/a/63986348/13745968">this post</a> that apparently says that these measures are not so necessary nor interfere so much in the final result.</p>
  <p>The arguments raised seemed to me quite convincing, but someone would know how to explain better, perhaps with a bibliographic reference, why is it not so necessary to use these techniques?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-25T18:55:03.197Z
Comments.Text: Tokenizing is a part of pre-processing.
Comments.UserDisplayName: Abhishek Verma
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-26T06:04:36.803Z
Comments.Text: @AbhishekVerma, these models already do tokenizing, indeed, but how does Transformers work so that a pre-processing step is little or nothing necessary?
Comments.UserDisplayName: Higo Felipe Silva Pires
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-27T13:34:45.420Z
Comments.Text: Why would you want to preprocess instead of using the raw data? Removing stop words, stemming, etc, removes information. If the raw data is given it can learn where to put its attention.
Comments.UserDisplayName: Isbister
Tags: <natural-language-processing><pytorch><transformer><data-preprocessing>
Title: Why (not) using pre-processing before using Transformer models?
UserDisplayName: Higo Felipe Silva Pires
ThreadText: # Why (not) using pre-processing before using Transformer models?
  
  (Asked by: username_0 on 2021-03-25)
  
  Regarding the use of pre-processing techniques before using Transformers models, I read [this post](https://stackoverflow.com/a/63986348/13745968) that apparently says that these measures are not so necessary nor interfere so much in the final result.
  
  The arguments raised seemed to me quite convincing, but someone would know how to explain better, perhaps with a bibliographic reference, why is it not so necessary to use these techniques?
  
  **Tags:** natural-language-processing, pytorch, transformer, data-preprocessing
  
  > **Comments:**
  > **username_1**: Tokenizing is a part of pre-processing.  
  > **username_0**: @username_1, these models already do tokenizing, indeed, but how does Transformers work so that a pre-processing step is little or nothing necessary?  
  > **username_2**: Why would you want to preprocess instead of using the raw data? Removing stop words, stemming, etc, removes information. If the raw data is given it can learn where to put its attention.  
  
  ---
  
  ## 0 answers
  
  