Answers.Body: <p>Imagine the tensor as a some generalized <span class="math-container">$n$</span>-dimensional hyperrectangle sliced into <span class="math-container">$n$</span>-dimensional hypercubes. Each element of the tensor is labeled by the position along the given axis, say <span class="math-container">$(x_1, x_2, \ldots)$</span>.</p>
  <p>Axis is not a property of tensor, rather the tensor is embedded in a <span class="math-container">$n$</span>-dimensional space, where the axes are chosen along the sides of the hyperrectangle corresponding to the tensor.</p>
  <p>There are many operations, that can be applied axiswise to tensor. Several examples:</p>
  <ul>
  <li>Mean along the axis (choose <span class="math-container">$0$</span> without loss of generality). Given the <span class="math-container">$n$</span>-dimensional tensor <span class="math-container">$x_{i_1, i_2 \ldots i_n}$</span> , the result will be <span class="math-container">$n-1$</span>-dimensional tensor <span class="math-container">$\frac{1}{N_0} \sum_{i_1} x_{i_1 i_2 \ldots i_n}$</span>, where <span class="math-container">$N_0$</span> is the number of elements of the tensor along the <span class="math-container">$0$</span>-th axis (height).</li>
  <li>Standard deviation along the axis. For each index <span class="math-container">$i_2 \ldots i_n$</span>, calculate <span class="math-container">$\sqrt{\frac{1}{N_0} \sum_{i_1}  (\bar{x}_{i_2 \ldots i_n} - x_{i_1 i_2 \ldots i_n})^2}$</span>, where <span class="math-container">$\bar{x}$</span> is the mean from previous point, and the result will be again <span class="math-container">$n-1$</span>-dimensional tensor.</li>
  </ul>
  <p>For your example <span class="math-container">$a$</span> is a <span class="math-container">$2$</span>-dimensional tensor with <span class="math-container">$2$</span> axes. <span class="math-container">$0$</span>-th axis corresponds to rows, <span class="math-container">$1$</span>-st axis corresponds to columns.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-29T04:27:46.707Z
Answers.LastActivityDate: 2022-06-16T20:29:58.093Z
Answers.LastEditDate: 2022-06-16T20:29:58.093Z
Answers.UserDisplayName: spiridon_the_sun_rotator
Answers.Body: <p><strong>In machine learning</strong>, a tensor is a multi-dimensional array (i.e. a generalization of a matrix to more than 2 dimensions), which has some properties, such as the number of dimensions or the shape, and to which you can apply operations (for example, you can take the mean of all elements across all dimensions). So, a scalar is a 0-d tensor (no dimensions), a vector is a 1d tensor (1 dimension), a matrix a 2d tensor (2 dimensions), a cube a 3d tensor, and so on.</p>
  <p>The name TensorFlow really comes from the fact that TensorFlow manipulates tensors all the time. The <a href="https://www.tensorflow.org/guide/tensor" rel="nofollow noreferrer">documentation of TensorFlow</a> describes tensors as follows</p>
  <blockquote>
  <p>Tensors are multi-dimensional arrays with a uniform type (called a dtype)</p>
  <p>If you're familiar with NumPy, tensors are (kind of) like np.arrays.</p>
  </blockquote>
  <p>In this context, you can think of the <strong>axis</strong> as an abstraction for dealing with or manipulating the dimensions (aka shape) of a tensor or to apply certain operations &quot;across dimensions&quot;. It's not exactly a synonym for dimension because, in some software libraries (e.g. Keras and NumPy), <a href="https://stackoverflow.com/q/47435526">you can also pass special arguments such as <code>-1</code> to the <code>axis</code> parameter</a>, which does not correspond to any dimension (there's no <span class="math-container">$-1$</span> dimension of a tensor). It's a way to apply certain operations to the tensor only across e.g. the first dimension (axis 0) or the second dimension (axis 2), and so on.</p>
  <p>For example, let's say you have a matrix (2d tensor) <span class="math-container">$A$</span>. You want to compute the average (aka mean) of the rows, i.e. sum all elements of row <span class="math-container">$i$</span>, then divide by the number of elements (which corresponds to the number of columns in row <span class="math-container">$i$</span>), and do this for all rows <span class="math-container">$i=1, \dots, K$</span>. You can do this in NumPy by specifying that you want to apply the mean operation to <code>axis=1</code>. So, after this operation, you will get a 1d tensor (a vector) with <span class="math-container">$K$</span> elements (i.e. the numbers of rows of the original matrix): this is what I mean by &quot;across dimensions&quot;.</p>
  <p>Here's a NumPy example that illustrates the concept.</p>
  <pre><code>import numpy as np
  
  A = np.array([[1, 2, 3],
                [1, 1, 1],
                [0, 1, -1]])
  
  # (1 + 2 + 3 + 1 + 1 + 1 + 0 + 1 + (-1)) / 9
  m = np.mean(A)
  print(&quot;Mean of all elements of A =&quot;, m)
  
  # [(1 + 2 + 3)/3, (1 + 1 + 1)/3, (0 + 1 + (-1))/3]
  m1 = np.mean(A, axis=1)
  print(&quot;Mean of each row =&quot;, m1)
  
  # [(1 + 1 + 0)/3, (2 + 1 + 1)/3, (3 + 1 + (-1))/3]
  m2 = np.mean(A, axis=0)
  print(&quot;Mean of each column =&quot;, m2)
  </code></pre>
  <p>Why would you want to compute (in this case) the mean of each row? Because, for example, each row <span class="math-container">$i$</span> may correspond to some data associated with a user <span class="math-container">$i$</span>, so you may want to compute e.g. the average salary for each user for all months (in this case, there are only 3 columns, so 3 months).</p>
  <p>Tensors did not originate in machine learning. In fact, in mathematics, <a href="https://en.wikipedia.org/wiki/Tensor" rel="nofollow noreferrer">tensors are well-known objects</a>, which have some properties, which may be different than the properties associated with the tensors implemented in libraries like TensorFlow, which are really just multi-dimensional arrays with the necessary properties and methods for machine/deep learning. Tensors are also everywhere in quantum computing.</p>
  <p><a href="https://ai.stackexchange.com/a/22202/2444">This answer</a> that I wrote a while ago could also be useful.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-30T16:27:27.797Z
Answers.LastActivityDate: 2021-07-30T16:35:00.583Z
Answers.LastEditDate: 2021-07-30T16:35:00.583Z
Answers.UserDisplayName: nbro
Body: <p>Tensor is an ordered collection of elements. The elements are generally real numbers. Tensors are used in deep learning for storing data.</p>
  <p>There is a wide usage of the word &quot;axis&quot; related to tensor. Axes are not the same as indices, which are used to access the elements of a tensor. An axis is not the same as an element of a tensor.</p>
  <p>What exactly is an axis in a tensor? Is it also a (sub-)tensor obtained from the actual tensor? Or is it any other indexing mechanism? If yes, why it is used?</p>
  <p>Suppose <span class="math-container">$a =[[1, 2, 3, 4],[5, 6, 7, 8],[9, 10, 11, 12],[13, 14, 15, 16]]$</span> is a tensor. Then what does axis do for <span class="math-container">$a$</span>?</p>
  
Tags: <deep-learning><terminology><math><definitions><tensor>
Title: What is meant by an axis of a tensor?
UserDisplayName: hanugm
ThreadText: # What is meant by an axis of a tensor?
  
  (Asked by: username_0 on 2021-07-28)
  
  Tensor is an ordered collection of elements. The elements are generally real numbers. Tensors are used in deep learning for storing data.
  
  There is a wide usage of the word "axis" related to tensor. Axes are not the same as indices, which are used to access the elements of a tensor. An axis is not the same as an element of a tensor.
  
  What exactly is an axis in a tensor? Is it also a (sub-)tensor obtained from the actual tensor? Or is it any other indexing mechanism? If yes, why it is used?
  
  Suppose $a =[[1, 2, 3, 4],[5, 6, 7, 8],[9, 10, 11, 12],[13, 14, 15, 16]]$ is a tensor. Then what does axis do for $a$?
  
  **Tags:** deep-learning, terminology, math, definitions, tensor
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2021-07-30. Score: 3 | Accepted answer
  
  **In machine learning**, a tensor is a multi-dimensional array (i.e. a generalization of a matrix to more than 2 dimensions), which has some properties, such as the number of dimensions or the shape, and to which you can apply operations (for example, you can take the mean of all elements across all dimensions). So, a scalar is a 0-d tensor (no dimensions), a vector is a 1d tensor (1 dimension), a matrix a 2d tensor (2 dimensions), a cube a 3d tensor, and so on.
  
  The name TensorFlow really comes from the fact that TensorFlow manipulates tensors all the time. The [documentation of TensorFlow](https://www.tensorflow.org/guide/tensor) describes tensors as follows
  
  > Tensors are multi-dimensional arrays with a uniform type (called a dtype)
  >
  > If you're familiar with NumPy, tensors are (kind of) like np.arrays.
  
  In this context, you can think of the **axis** as an abstraction for dealing with or manipulating the dimensions (aka shape) of a tensor or to apply certain operations "across dimensions". It's not exactly a synonym for dimension because, in some software libraries (e.g. Keras and NumPy), [you can also pass special arguments such as `-1` to the `axis` parameter](https://stackoverflow.com/q/47435526), which does not correspond to any dimension (there's no $-1$ dimension of a tensor). It's a way to apply certain operations to the tensor only across e.g. the first dimension (axis 0) or the second dimension (axis 2), and so on.
  
  For example, let's say you have a matrix (2d tensor) $A$. You want to compute the average (aka mean) of the rows, i.e. sum all elements of row $i$, then divide by the number of elements (which corresponds to the number of columns in row $i$), and do this for all rows $i=1, \dots, K$. You can do this in NumPy by specifying that you want to apply the mean operation to `axis=1`. So, after this operation, you will get a 1d tensor (a vector) with $K$ elements (i.e. the numbers of rows of the original matrix): this is what I mean by "across dimensions".
  
  Here's a NumPy example that illustrates the concept.
  
  ```
  import numpy as np
  
  A = np.array([[1, 2, 3],
                [1, 1, 1],
                [0, 1, -1]])
  
  # (1 + 2 + 3 + 1 + 1 + 1 + 0 + 1 + (-1)) / 9
  m = np.mean(A)
  print("Mean of all elements of A =", m)
  
  # [(1 + 2 + 3)/3, (1 + 1 + 1)/3, (0 + 1 + (-1))/3]
  m1 = np.mean(A, axis=1)
  print("Mean of each row =", m1)
  
  # [(1 + 1 + 0)/3, (2 + 1 + 1)/3, (3 + 1 + (-1))/3]
  m2 = np.mean(A, axis=0)
  print("Mean of each column =", m2)
  
  ```
  
  Why would you want to compute (in this case) the mean of each row? Because, for example, each row $i$ may correspond to some data associated with a user $i$, so you may want to compute e.g. the average salary for each user for all months (in this case, there are only 3 columns, so 3 months).
  
  Tensors did not originate in machine learning. In fact, in mathematics, [tensors are well-known objects](https://en.wikipedia.org/wiki/Tensor), which have some properties, which may be different than the properties associated with the tensors implemented in libraries like TensorFlow, which are really just multi-dimensional arrays with the necessary properties and methods for machine/deep learning. Tensors are also everywhere in quantum computing.
  
  [This answer](https://ai.stackexchange.com/a/22202/2444) that I wrote a while ago could also be useful.
  
  
  ---
  
  ### Answer by username_1 on 2021-07-29. Score: 1
  
  Imagine the tensor as a some generalized $n$-dimensional hyperrectangle sliced into $n$-dimensional hypercubes. Each element of the tensor is labeled by the position along the given axis, say $(x\_1, x\_2, \ldots)$.
  
  Axis is not a property of tensor, rather the tensor is embedded in a $n$-dimensional space, where the axes are chosen along the sides of the hyperrectangle corresponding to the tensor.
  
  There are many operations, that can be applied axiswise to tensor. Several examples:
  
  * Mean along the axis (choose $0$ without loss of generality). Given the $n$-dimensional tensor $x\_{i\_1, i\_2 \ldots i\_n}$ , the result will be $n-1$-dimensional tensor $\frac{1}{N\_0} \sum\_{i\_1} x\_{i\_1 i\_2 \ldots i\_n}$, where $N\_0$ is the number of elements of the tensor along the $0$-th axis (height).
  * Standard deviation along the axis. For each index $i\_2 \ldots i\_n$, calculate $\sqrt{\frac{1}{N\_0} \sum\_{i\_1} (\bar{x}\_{i\_2 \ldots i\_n} - x\_{i\_1 i\_2 \ldots i\_n})^2}$, where $\bar{x}$ is the mean from previous point, and the result will be again $n-1$-dimensional tensor.
  
  For your example $a$ is a $2$-dimensional tensor with $2$ axes. $0$-th axis corresponds to rows, $1$-st axis corresponds to columns.
  
  
  ---
  
  