Answers.Body: <blockquote>
  <p>Do they mean the strides that are related to the CNN, pooling, etc., or are they referring to any other stride information?</p>
  </blockquote>
  <p>The stride referred to by the quote &quot;only plays with the size and stride information at the tensor level&quot; is referring to internal storage of tensors. Luckily in most normal conversations about AI logic you do not care about this, it has nothing to do with neural network implementation, but is a lower-level optimisation (compared to some other ways of implementing tensors that would not use an internal stride).</p>
  <p>Most general-purpose computer chips do not use multi-part addressing for data, such as <code>(x,y,z)</code>, but instead use a single memory location number. To construct a data structure to represent a 2-or-more dimensional tensor means combining more fundamental representations. There are a few different ways to do this. One might be an array of arrays, where the outer array contains pointers to memory addresses of inner arrays. However, a single array with some management numbers attached to it (typically at the beginning, or even as a separate data structure, so you can have multiple tensor views of the same data) has some advantages - some operations on the array are far more efficient. Reshaping and transposing are two operations that can be implemented far more efficiently when the storage is arranged in this way.</p>
  <p>So, along with the tensor cell data (the number inside each location), a typical tensor implementation will store information <em>separately</em> about how that data is structured. One useful piece of data is the stride, which is how many memory locations to add for each step in a particular direction in the n-indices view of the data. For example in a 2-indexed tensor with shape (6,8) where the data is stored in rows, then incrementing the column index will take a +1 step in memory location, whilst incrementing the row index will take a +8 step (the number of columns in a row). This is the row stride for that shape.</p>
  <p>Technically these two types of stride are conceptually very similar. The difference is the level within the code that they are implemented in. Usually for most work <em>using</em> a tensor library you only care about this &quot;internal&quot; stride happening because it gives you confidence that there is a low cost to a particular tensor operation, such as generating a different shaped view.</p>
  <p>Most importantly, unless you are working on your own tensor library, or digging into the internals of one using a low-level language (C), then you do not need to specify the stride values used. Instead you can take the quoted text as a declaration that a particular operation is implemented efficiently and you can use it in your own code with confidence that there is not a high CPU (or GPU) cost to using it.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-16T13:18:15.107Z
Answers.LastActivityDate: 2022-01-20T01:29:10.250Z
Answers.LastEditDate: 2022-01-20T01:29:10.250Z
Answers.UserDisplayName: Neil Slater
Body: <p>In convolutional neural networks, the convolution and pooling operations have a parameter known as <em>stride</em>, which decides the amount of jump the kernel needs to do on the input image. You can get more information regarding stride from follows taken from <a href="https://medium.com/@RaghavPrabhu/understanding-of-convolutional-neural-network-cnn-deep-learning-99760835f148" rel="nofollow noreferrer">here</a></p>
  <blockquote>
  <p>Stride is the number of pixels shifts over the input matrix. When the
  stride is 1 then we move the filters to 1 pixel at a time. When the
  stride is 2 then we move the filters to 2 pixels at a time and so on.</p>
  </blockquote>
  <p>But, I am not getting what does it mean by stride information of an image at the tensor level. Consider the <a href="https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf#page=103" rel="nofollow noreferrer">following paragraph</a> from the chapter named <strong>Real-world data representation using tensors</strong> from the textbook titled <a href="https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf" rel="nofollow noreferrer">Deep Learning with PyTorch</a> by <em>Eli Stevens et al.</em></p>
  <blockquote>
  <pre><code>img = torch.from_numpy(img_arr)
  out = img.permute(2, 0, 1)
  </code></pre>
  <p>We’ve seen this previously, but note that this operation does not make
  a copy of the tensor data. Instead, <code>out</code> uses the same underlying
  storage as <code>img</code> and <strong>only plays with the size and stride information
  at the tensor level.</strong> This is convenient because the operation is very
  cheap; but just as a heads-up: changing a pixel in <code>img</code> will lead to
  a change in <code>out</code>.</p>
  </blockquote>
  <p>It is mentioning about the stride information at the tensor level of an image. Do they mean the strides that are related to the CNN, pooling, etc., or are they referring to any other stride information?</p>
  
Tags: <deep-learning><convolutional-neural-networks><terminology><image-processing><stride>
Title: What is the stride information of an image referring here?
UserDisplayName: hanugm
ThreadText: # What is the stride information of an image referring here?
  
  (Asked by: username_0 on 2021-11-26)
  
  In convolutional neural networks, the convolution and pooling operations have a parameter known as *stride*, which decides the amount of jump the kernel needs to do on the input image. You can get more information regarding stride from follows taken from [here](https://medium.com/@RaghavPrabhu/understanding-of-convolutional-neural-network-cnn-deep-learning-99760835f148)
  
  > Stride is the number of pixels shifts over the input matrix. When the
  > stride is 1 then we move the filters to 1 pixel at a time. When the
  > stride is 2 then we move the filters to 2 pixels at a time and so on.
  
  But, I am not getting what does it mean by stride information of an image at the tensor level. Consider the [following paragraph](https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf#page=103) from the chapter named **Real-world data representation using tensors** from the textbook titled [Deep Learning with PyTorch](https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf) by *Eli Stevens et al.*
  
  > ```
  > img = torch.from_numpy(img_arr)
  > out = img.permute(2, 0, 1)
  >
  > ```
  >
  > We’ve seen this previously, but note that this operation does not make
  > a copy of the tensor data. Instead, `out` uses the same underlying
  > storage as `img` and **only plays with the size and stride information
  > at the tensor level.** This is convenient because the operation is very
  > cheap; but just as a heads-up: changing a pixel in `img` will lead to
  > a change in `out`.
  
  It is mentioning about the stride information at the tensor level of an image. Do they mean the strides that are related to the CNN, pooling, etc., or are they referring to any other stride information?
  
  **Tags:** deep-learning, convolutional-neural-networks, terminology, image-processing, stride
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-01-16. Score: 2 | Accepted answer
  
  > Do they mean the strides that are related to the CNN, pooling, etc., or are they referring to any other stride information?
  
  The stride referred to by the quote "only plays with the size and stride information at the tensor level" is referring to internal storage of tensors. Luckily in most normal conversations about AI logic you do not care about this, it has nothing to do with neural network implementation, but is a lower-level optimisation (compared to some other ways of implementing tensors that would not use an internal stride).
  
  Most general-purpose computer chips do not use multi-part addressing for data, such as `(x,y,z)`, but instead use a single memory location number. To construct a data structure to represent a 2-or-more dimensional tensor means combining more fundamental representations. There are a few different ways to do this. One might be an array of arrays, where the outer array contains pointers to memory addresses of inner arrays. However, a single array with some management numbers attached to it (typically at the beginning, or even as a separate data structure, so you can have multiple tensor views of the same data) has some advantages - some operations on the array are far more efficient. Reshaping and transposing are two operations that can be implemented far more efficiently when the storage is arranged in this way.
  
  So, along with the tensor cell data (the number inside each location), a typical tensor implementation will store information *separately* about how that data is structured. One useful piece of data is the stride, which is how many memory locations to add for each step in a particular direction in the n-indices view of the data. For example in a 2-indexed tensor with shape (6,8) where the data is stored in rows, then incrementing the column index will take a +1 step in memory location, whilst incrementing the row index will take a +8 step (the number of columns in a row). This is the row stride for that shape.
  
  Technically these two types of stride are conceptually very similar. The difference is the level within the code that they are implemented in. Usually for most work *using* a tensor library you only care about this "internal" stride happening because it gives you confidence that there is a low cost to a particular tensor operation, such as generating a different shaped view.
  
  Most importantly, unless you are working on your own tensor library, or digging into the internals of one using a low-level language (C), then you do not need to specify the stride values used. Instead you can take the quoted text as a declaration that a particular operation is implemented efficiently and you can use it in your own code with confidence that there is not a high CPU (or GPU) cost to using it.
  
  
  ---
  
  