Answers.Body: <h2>Short answer</h2>
  <p>Shuffling affects learning (i.e. the updates of the parameters of the model), but, during testing or validation, you are not learning. So, it should not make any difference whether you shuffle or not the test or validation data (unless you are computing some metric that depends on the order of the samples), given that you will not be computing any gradient, but just the loss or some metric/measure like the accuracy, which is not sensitive to the order or the samples you use to compute it. However, the specific samples that you use affects the computation of the loss and these quality metrics. So, how you split your original data into training, validation and test datasets affects the computation of the loss and metrics during validation and testing.</p>
  <h2>Long answer</h2>
  <p>Let me describe how gradient descent (GD) and <em>stochastic</em> gradient descent (SGD) are used to train machine learning models and, in particular, neural networks.</p>
  <h3>Gradient descent (GD)</h3>
  <p>When training ML models with GD, you have a loss (aka cost) function <span class="math-container">$L(\theta; D)$</span> (e.g. the cross-entropy or mean squared error) that you are trying to minimize, where <span class="math-container">$\theta \in \mathbb{R}^m$</span> is a vector of parameters of your model and <span class="math-container">$D$</span> is your labeled training dataset.</p>
  <p>To minimize this function using GD, you compute the <a href="https://ai.stackexchange.com/a/20380/2444"><em>gradient</em> of your loss function <span class="math-container">$L(\theta; D)$</span> with respect to the parameters of your model <span class="math-container">$\theta$</span> given the training samples</a>. Let's denote this gradient by <span class="math-container">$\nabla_\theta L(\theta; D) \in \mathbb{R}^m$</span>. Then we perform a step of gradient descent</p>
  <p><span class="math-container">$$
  \theta \leftarrow \theta - \alpha \nabla_\theta L(\theta; D) \label{1}\tag{1}
  $$</span></p>
  <h3>Stochastic gradient descent (SGD)</h3>
  <p>You can also minimize <span class="math-container">$L$</span> using <em>stochastic</em> gradient descent, i.e. you compute an approximate (or <em>stochastic</em>) version of <span class="math-container">$ \nabla_\theta L(\theta; D)$</span>, which we can denote as <span class="math-container">$\tilde{\nabla}_\theta L(\theta; B) \approx \nabla_\theta L(\theta; D)$</span>, which is typically computed with a subset of <span class="math-container">$B$</span> of your training dataset <span class="math-container">$D$</span>, i.e. <span class="math-container">$B \subset D$</span> and <span class="math-container">$|B| &lt; |D|$</span>. The step of SGD is exactly the same as the step of GD, but we use <span class="math-container">$\tilde{\nabla}_\theta L(\theta; B)$</span></p>
  <p><span class="math-container">$$
  \theta \leftarrow \theta - \alpha \tilde{\nabla}_\theta L(\theta; B) \label{2}\tag{2}
  $$</span>
  If we split <span class="math-container">$D$</span> into <span class="math-container">$k$</span> subsets (or batches) <span class="math-container">$B_i$</span>, for <span class="math-container">$i=1, \dots, k$</span> (and these subsets usually have the same size, i.e. <span class="math-container">$|B_i| = |B_j|, \forall i$</span>, apart from one of them, which may contain fewer samples), then the SGD step needs to be performed <span class="math-container">$k$</span> times, in order to go through all training samples.</p>
  <h3>Sampling, shuffling, and convergence</h3>
  <p>Given that <span class="math-container">$\tilde{\nabla}_\theta L(\theta; B_i) \approx \nabla_\theta L(\theta; D), \forall i$</span>, it should be clear that the way you split the samples into batches can affect learning (i.e. the updates of the parameters).</p>
  <p>For instance, you could consider your dataset <span class="math-container">$D$</span> as an ordered sequence/list, and just split it into <span class="math-container">$k$</span> sub-sequences. Without shuffling this ordered sequence before splitting, you will always get the same batches, which means that, if there's some information associated with the specific ordering of this sequence, then it may bias the learning process. That's one of the reasons why you may want to shuffle the data.</p>
  <p>So, you could <em>uniformly</em> choose samples from <span class="math-container">$D$</span> to create your batches <span class="math-container">$B_i$</span> (and this is a way of shuffling, in the sense that you will be uniformly building these batches at random), but you can also sample differently and you could also re-use the same samples in different batches (i.e. sampling with replacement). Of course, all these approaches can affect how learning proceeds.</p>
  <p>Typically, when analyzing the convergence properties of SGD, you require that your samples are <a href="https://ai.stackexchange.com/a/10844/2444">i.i.d.</a> and that the learning rate <span class="math-container">$\alpha$</span> satisfies some conditions (<a href="https://en.wikipedia.org/wiki/Stochastic_approximation" rel="noreferrer">the Robbins–Monro conditions</a>). If that's not the case, then SGD may not converge to the correct answer. That's why sampling or shuffling can play an important role in SGD.</p>
  <h3>Testing and validation</h3>
  <p>During testing or validation, you are just computing the loss or some metric (like the accuracy) and not a stochastic gradient (i.e. you are not updating the parameters, by definition: you just do it during training). The way you compute the loss or accuracy should not be sensitive to the order of the samples, so shuffling should not affect the computation of the loss or accuracy. For instance, if you use the mean squared error, then you will need to compute</p>
  <p><span class="math-container">\begin{align}
  L(\theta; D_\text{test}) 
  &amp;= \operatorname {MSE} \\
  &amp;= {\frac {1}{n}}\sum _{i=1}^{n}(f_\theta(x_i)-{\hat {y_{i}}})^{2}\\
  &amp;= {\frac {1}{n}}\sum _{i=1}^{n}(y_{i}-{\hat {y_{i}}})^{2}
  \end{align}</span></p>
  <p>where</p>
  <ul>
  <li><span class="math-container">$f_\theta$</span> is your ML model</li>
  <li><span class="math-container">$x_i$</span> is the <span class="math-container">$i$</span>th input</li>
  <li><span class="math-container">$y_{i}$</span> is the true label for input <span class="math-container">$x_i$</span></li>
  <li><span class="math-container">$\hat {y_{i}}$</span> is the output of the model</li>
  <li><span class="math-container">$n$</span> is the number of samples you use to compute the MSE</li>
  </ul>
  <p>This is an average, so it doesn't really matter whether you shuffle or not. Of course, it matters which samples you use though!</p>
  <h2>Further reading</h2>
  <p><a href="https://stats.stackexchange.com/q/245502/82135">Here</a> you can find some informal answers to the question &quot;Why do we shuffle the training data while training a neural network?&quot;. There are other papers that partially answer this and/or other related questions more formally, such as <a href="http://proceedings.mlr.press/v97/haochen19a/haochen19a.pdf" rel="noreferrer">this</a> or <a href="https://www.sciencedirect.com/science/article/pii/S0925231219300578" rel="noreferrer">this</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-03T12:02:58.430Z
Answers.LastActivityDate: 2020-11-03T13:57:53.987Z
Answers.LastEditDate: 2020-11-03T13:57:53.987Z
Answers.UserDisplayName: nbro
Body: <p>When training machine learning models (e.g. neural networks) with stochastic gradient descent, it is common practice to (uniformly) shuffle the <em>training</em> data into batches/sets of different samples from different classes. Should we also shuffle the <em>test</em> dataset?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-03T05:46:22.117Z
Comments.Text: There is no point to shuffle the test or validation data. It's only done in the training time.
Comments.UserDisplayName: Innat
Tags: <machine-learning><training><datasets><stochastic-gradient-descent><testing>
Title: Should we also shuffle the test dataset when training with SGD?
UserDisplayName: SpiderRico
ThreadText: # Should we also shuffle the test dataset when training with SGD?
  
  (Asked by: username_0 on 2020-11-03)
  
  When training machine learning models (e.g. neural networks) with stochastic gradient descent, it is common practice to (uniformly) shuffle the *training* data into batches/sets of different samples from different classes. Should we also shuffle the *test* dataset?
  
  **Tags:** machine-learning, training, datasets, stochastic-gradient-descent, testing
  
  > **Comments:**
  > **username_1**: There is no point to shuffle the test or validation data. It's only done in the training time.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-11-03. Score: 9 | Accepted answer
  
  ## Short answer
  
  Shuffling affects learning (i.e. the updates of the parameters of the model), but, during testing or validation, you are not learning. So, it should not make any difference whether you shuffle or not the test or validation data (unless you are computing some metric that depends on the order of the samples), given that you will not be computing any gradient, but just the loss or some metric/measure like the accuracy, which is not sensitive to the order or the samples you use to compute it. However, the specific samples that you use affects the computation of the loss and these quality metrics. So, how you split your original data into training, validation and test datasets affects the computation of the loss and metrics during validation and testing.
  
  ## Long answer
  
  Let me describe how gradient descent (GD) and *stochastic* gradient descent (SGD) are used to train machine learning models and, in particular, neural networks.
  
  ### Gradient descent (GD)
  
  When training ML models with GD, you have a loss (aka cost) function $L(\theta; D)$ (e.g. the cross-entropy or mean squared error) that you are trying to minimize, where $\theta \in \mathbb{R}^m$ is a vector of parameters of your model and $D$ is your labeled training dataset.
  
  To minimize this function using GD, you compute the [*gradient* of your loss function $L(\theta; D)$ with respect to the parameters of your model $\theta$ given the training samples](https://ai.stackexchange.com/a/20380/2444). Let's denote this gradient by $\nabla\_\theta L(\theta; D) \in \mathbb{R}^m$. Then we perform a step of gradient descent
  
  $$
  \theta \leftarrow \theta - \alpha \nabla\_\theta L(\theta; D) \label{1}\tag{1}
  $$
  
  ### Stochastic gradient descent (SGD)
  
  You can also minimize $L$ using *stochastic* gradient descent, i.e. you compute an approximate (or *stochastic*) version of $ \nabla\_\theta L(\theta; D)$, which we can denote as $\tilde{\nabla}\_\theta L(\theta; B) \approx \nabla\_\theta L(\theta; D)$, which is typically computed with a subset of $B$ of your training dataset $D$, i.e. $B \subset D$ and $|B| < |D|$. The step of SGD is exactly the same as the step of GD, but we use $\tilde{\nabla}\_\theta L(\theta; B)$
  
  $$
  \theta \leftarrow \theta - \alpha \tilde{\nabla}\_\theta L(\theta; B) \label{2}\tag{2}
  $$
  If we split $D$ into $k$ subsets (or batches) $B\_i$, for $i=1, \dots, k$ (and these subsets usually have the same size, i.e. $|B\_i| = |B\_j|, \forall i$, apart from one of them, which may contain fewer samples), then the SGD step needs to be performed $k$ times, in order to go through all training samples.
  
  ### Sampling, shuffling, and convergence
  
  Given that $\tilde{\nabla}\_\theta L(\theta; B\_i) \approx \nabla\_\theta L(\theta; D), \forall i$, it should be clear that the way you split the samples into batches can affect learning (i.e. the updates of the parameters).
  
  For instance, you could consider your dataset $D$ as an ordered sequence/list, and just split it into $k$ sub-sequences. Without shuffling this ordered sequence before splitting, you will always get the same batches, which means that, if there's some information associated with the specific ordering of this sequence, then it may bias the learning process. That's one of the reasons why you may want to shuffle the data.
  
  So, you could *uniformly* choose samples from $D$ to create your batches $B\_i$ (and this is a way of shuffling, in the sense that you will be uniformly building these batches at random), but you can also sample differently and you could also re-use the same samples in different batches (i.e. sampling with replacement). Of course, all these approaches can affect how learning proceeds.
  
  Typically, when analyzing the convergence properties of SGD, you require that your samples are [i.i.d.](https://ai.stackexchange.com/a/10844/2444) and that the learning rate $\alpha$ satisfies some conditions ([the Robbins–Monro conditions](https://en.wikipedia.org/wiki/Stochastic_approximation)). If that's not the case, then SGD may not converge to the correct answer. That's why sampling or shuffling can play an important role in SGD.
  
  ### Testing and validation
  
  During testing or validation, you are just computing the loss or some metric (like the accuracy) and not a stochastic gradient (i.e. you are not updating the parameters, by definition: you just do it during training). The way you compute the loss or accuracy should not be sensitive to the order of the samples, so shuffling should not affect the computation of the loss or accuracy. For instance, if you use the mean squared error, then you will need to compute
  
  \begin{align}
  L(\theta; D\_\text{test})
  &= \operatorname {MSE} \\
  &= {\frac {1}{n}}\sum \_{i=1}^{n}(f\_\theta(x\_i)-{\hat {y\_{i}}})^{2}\\
  &= {\frac {1}{n}}\sum \_{i=1}^{n}(y\_{i}-{\hat {y\_{i}}})^{2}
  \end{align}
  
  where
  
  * $f\_\theta$ is your ML model
  * $x\_i$ is the $i$th input
  * $y\_{i}$ is the true label for input $x\_i$
  * $\hat {y\_{i}}$ is the output of the model
  * $n$ is the number of samples you use to compute the MSE
  
  This is an average, so it doesn't really matter whether you shuffle or not. Of course, it matters which samples you use though!
  
  ## Further reading
  
  [Here](https://stats.stackexchange.com/q/245502/82135) you can find some informal answers to the question "Why do we shuffle the training data while training a neural network?". There are other papers that partially answer this and/or other related questions more formally, such as [this](http://proceedings.mlr.press/v97/haochen19a/haochen19a.pdf) or [this](https://www.sciencedirect.com/science/article/pii/S0925231219300578).
  
  
  ---
  
  