Answers.Body: <p>Yes, sure, <a href="https://ch.mathworks.com/help/deeplearning/deep-learning-data-management-and-preprocessing.html" rel="nofollow noreferrer">data pre-processing</a> is also done in deep learning. For example, we often normalize (or scale) the inputs to neural networks. If the inputs are images, we often resize them so that they all have the same dimensions. Of course, the pre-processing step that you apply depends on your data, neural network, and task.</p>
  <p><a href="https://github.com/tensorflow/models/blob/master/official/nlp/transformer/data_pipeline.py" rel="nofollow noreferrer">Here</a> or <a href="https://github.com/tensorflow/models/blob/master/official/vision/image_classification/mnist_main.py" rel="nofollow noreferrer">here</a> are two examples of implementations that perform a pre-processing step (normalization in the second case). You can find more explanations and examples <a href="https://ch.mathworks.com/help/deeplearning/deep-learning-data-management-and-preprocessing.html" rel="nofollow noreferrer">here</a> and probably <a href="https://github.com/tensorflow/models/tree/master/official" rel="nofollow noreferrer">here</a> too.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-14T18:41:29.950Z
Answers.LastActivityDate: 2021-01-14T18:56:56.227Z
Answers.LastEditDate: 2021-01-14T18:56:56.227Z
Answers.UserDisplayName: nbro
Answers.Body: <p>Adding to nbro's solution, the less you have to normalize/balance/preprocess/augment, the better, e.g. because then you know for sure that the accuracy is the achievement of the model rather than data combing. For example, if you can achieve the same accuracy using two approaches (e.g. with the image dataset):</p>
  <ol>
  <li>for each image, subtract global mean, divide by global standard deviation</li>
  <li>1)+random flips, random crops, color jitter, etc,</li>
  </ol>
  <p>then 1), if you can achieve a comparable accuracy, is a better solution, as the model is more general. The same refers to the balancing of the data - if you can train a good model without it, it's an additional strength.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-14T22:43:21.250Z
Answers.Text: I wouldn't say that the model is "more general". I would say that the model has learned from a different range of inputs and possibly a more complicated function. So, I'm not really sure about the correctness of this answer, so I would like to read some reliable research paper that goes in the direction of what you're saying.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-15T11:12:44.670Z
Answers.Text: This can be verified on running the model on the out-of-sample data
Answers.UserDisplayName: Alex
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-15T11:14:51.027Z
Answers.Text: Out-of-sample data, of course, can cause problems in both cases (normalization or not). So, would you care to show some code/research that illustrates the issue and really supports your claim?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-14T19:46:34.503Z
Answers.LastActivityDate: 2021-01-14T19:46:34.503Z
Answers.UserDisplayName: Alex
Answers.Body: <p>Preprocessing in deep learning is about how to make data compatible to the model architecture's input tensor shape and output tensor shape.</p>
  <p>So the main critical aspect of preprocessing in deep learning is how to encode the data into the tensor with a specified shape of model input so that it can be fed into the deep learning model.</p>
  <p>As you already know, deep learning is intended to avoid any human intervention or feature engineering (opposite what traditional machine learning does). Therefore, normalization and feature scaling in deep learning are optional things, it's because they are considered as feature engineering. Unless this is the requirement to make compatibility shape of model input. Mainly if your data contains physical units that represent the real world. You can just let the model train that real-world thing.</p>
  <p>When you encode the data to the tensor, you might lose some information such as metadata or index mapping, this is common mistake. You should keep the metadata or hard coded it into code.</p>
  <p>The metadata supposed to decode the tensor back to the data that represent original data in post-processing step or inferring process.</p>
  <p>So, this is the overall deep learning pipeline:</p>
  <h1>Training</h1>
  <p><a href="https://i.sstatic.net/o0fM5HA4.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/o0fM5HA4.png" alt="enter image description here" /></a></p>
  <h1>Inference</h1>
  <p><a href="https://i.sstatic.net/9nU7uB1K.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/9nU7uB1K.png" alt="enter image description here" /></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-18T09:06:55.623Z
Answers.LastActivityDate: 2024-07-08T20:57:48.073Z
Answers.LastEditDate: 2024-07-08T20:57:48.073Z
Answers.UserDisplayName: Muhammad Ikhwan Perwira
Body: <p>I'm new to deep learning. I wanted to know: do we use pre-processing in deep learning? Or it is only used in machine learning. I searched for it and its methods on the internet, but I didn't find a suitable answer.</p>
  
Tags: <machine-learning><deep-learning><data-preprocessing>
Title: Is pre-processing used in deep learning?
UserDisplayName: Pablo
ThreadText: # Is pre-processing used in deep learning?
  
  (Asked by: username_0 on 2021-01-14)
  
  I'm new to deep learning. I wanted to know: do we use pre-processing in deep learning? Or it is only used in machine learning. I searched for it and its methods on the internet, but I didn't find a suitable answer.
  
  **Tags:** machine-learning, deep-learning, data-preprocessing
  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_1 on 2021-01-14. Score: 1
  
  Yes, sure, [data pre-processing](https://ch.mathworks.com/help/deeplearning/deep-learning-data-management-and-preprocessing.html) is also done in deep learning. For example, we often normalize (or scale) the inputs to neural networks. If the inputs are images, we often resize them so that they all have the same dimensions. Of course, the pre-processing step that you apply depends on your data, neural network, and task.
  
  [Here](https://github.com/tensorflow/models/blob/master/official/nlp/transformer/data_pipeline.py) or [here](https://github.com/tensorflow/models/blob/master/official/vision/image_classification/mnist_main.py) are two examples of implementations that perform a pre-processing step (normalization in the second case). You can find more explanations and examples [here](https://ch.mathworks.com/help/deeplearning/deep-learning-data-management-and-preprocessing.html) and probably [here](https://github.com/tensorflow/models/tree/master/official) too.
  
  
  ---
  
  ### Answer by username_2 on 2021-01-14. Score: 0
  
  Adding to username_1's solution, the less you have to normalize/balance/preprocess/augment, the better, e.g. because then you know for sure that the accuracy is the achievement of the model rather than data combing. For example, if you can achieve the same accuracy using two approaches (e.g. with the image dataset):
  
  1. for each image, subtract global mean, divide by global standard deviation
  2. 1)+random flips, random crops, color jitter, etc,
  
  then 1), if you can achieve a comparable accuracy, is a better solution, as the model is more general. The same refers to the balancing of the data - if you can train a good model without it, it's an additional strength.
  
  > **Comments:**
  > **username_1**: I wouldn't say that the model is "more general". I would say that the model has learned from a different range of inputs and possibly a more complicated function. So, I'm not really sure about the correctness of this answer, so I would like to read some reliable research paper that goes in the direction of what you're saying.  
  > **username_2**: This can be verified on running the model on the out-of-sample data  
  > **username_1**: Out-of-sample data, of course, can cause problems in both cases (normalization or not). So, would you care to show some code/research that illustrates the issue and really supports your claim?  
  
  ---
  
  ### Answer by username_3 on 2024-06-18. Score: 0
  
  Preprocessing in deep learning is about how to make data compatible to the model architecture's input tensor shape and output tensor shape.
  
  So the main critical aspect of preprocessing in deep learning is how to encode the data into the tensor with a specified shape of model input so that it can be fed into the deep learning model.
  
  As you already know, deep learning is intended to avoid any human intervention or feature engineering (opposite what traditional machine learning does). Therefore, normalization and feature scaling in deep learning are optional things, it's because they are considered as feature engineering. Unless this is the requirement to make compatibility shape of model input. Mainly if your data contains physical units that represent the real world. You can just let the model train that real-world thing.
  
  When you encode the data to the tensor, you might lose some information such as metadata or index mapping, this is common mistake. You should keep the metadata or hard coded it into code.
  
  The metadata supposed to decode the tensor back to the data that represent original data in post-processing step or inferring process.
  
  So, this is the overall deep learning pipeline:
  
  # Training
  
  [![enter image description here](https://i.sstatic.net/o0fM5HA4.png)](https://i.sstatic.net/o0fM5HA4.png)
  
  # Inference
  
  [![enter image description here](https://i.sstatic.net/9nU7uB1K.png)](https://i.sstatic.net/9nU7uB1K.png)
  
  
  ---
  
  