Answers.Body: <p>An isomorphism <span class="math-container">$T$</span> between vector spaces <span class="math-container">$V$</span> and <span class="math-container">$W$</span> over the same field <span class="math-container">$K$</span> (for example, <span class="math-container">$K = \mathbb{R}$</span>) is defined as a <a href="http://www2.clarku.edu/%7Edjoyce/ma130/isomorphism.pdf" rel="nofollow noreferrer"><strong>bijective</strong></a> (i.e. 1-to-1 and onto, which makes a bijection an invertible function) transformation (a function) that preserves the 2 main properties</p>
  <ol>
  <li>scalar multiplication: <span class="math-container">$T(c\mathbf {u} )=cT(\mathbf {u} )$</span>, where <span class="math-container">$c \in K$</span> and <span class="math-container">$\mathbf {u} \in V$</span></li>
  <li>vector addition: <span class="math-container">$T(\mathbf {u} +\mathbf {v} )=T(\mathbf {u} )+T(\mathbf {v})$</span>, where <span class="math-container">$\mathbf {u}, \mathbf {v} \in V$</span></li>
  </ol>
  <p>Linear maps preserve these 2 properties, but not all linear maps are bijective, so not all linear maps are isomorphisms. In fact, linear maps that are bijective are called linear isomorphisms. You can denote an isomorphism between <span class="math-container">$V$</span> and <span class="math-container">$W$</span> as follows (to emphasize that it's not just a linear map): <span class="math-container">$T: V {\overset {\sim }{\to }} W$</span>.</p>
  <p>The fact that an isomorphism is invertible means that there exists a function <span class="math-container">$T^{-1}$</span> that transforms any element of <span class="math-container">$W$</span> to any element of <span class="math-container">$V$</span>.</p>
  <p>So, if there's an isomorphism <span class="math-container">$T$</span> between <span class="math-container">$V = \mathbb{R}^{n \times m}$</span> and <span class="math-container">$W = \mathbb{R}^{nm}$</span> (note that I am not proving it here, but it's probably not complicated to prove), then it implies that, for each <span class="math-container">$\mathbf {v}_W \in \mathbb{R}^{nm}$</span>, you can use <span class="math-container">$T^{-1}: \mathbb{R}^{nm} {\overset {\sim }{\to }} \mathbb{R}^{n \times m}$</span> to retrieve the corresponding vector <span class="math-container">$\mathbf {v}_V \in \mathbb{R}^{n \times m}$</span> (not that, in the context of vector spaces, matrices are also vectors, which is just the name used to denote objects in <em>vector</em> spaces).</p>
  <p>This should prove why, if reshaping is an isomorphism, then you can perform operations first on <span class="math-container">$\mathbf {v}_W$</span> and then go back to <span class="math-container">$\mathbf {v}_V$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-03T12:17:27.340Z
Answers.Text: To prove reshaping is an isomorphism, you need to prove it's bijective and show that it preserves scalar and vector addition. This shouldn't be too difficult (I think, at least if you had one course in linear algebra), but if you have problems, let me know and I can try to help.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-03T12:07:55.173Z
Answers.LastActivityDate: 2021-12-03T12:07:55.173Z
Answers.UserDisplayName: nbro
Answers.Body: <p>I think the textbook is making it unnecessarily complicated. <span class="math-container">$\mathbb{R}^{n\times m}$</span> and <span class="math-container">$\mathbb{R}^{nm}$</span> both represent the same vector space, a real vector space of dimension <span class="math-container">$nm$</span>. (So clearly they are isomorphic because they are the same.)</p>
  <p>The interpretation of them being either a space of vectors or a space of matrices is an artificial distinction in this context.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-30T15:25:29.830Z
Answers.Text: $\mathbb{R}^{n\times m}$ and $\mathbb{R}^{nm}$ are different spaces: matrices and vectors. Each real value in a matrix has a row and a column. Each value in a vector has just an index. There are two standard mappings between them: row-major and column-major, plus many uncommon mappings. Therefore they are not "the same" space (although they are isomorphic.) You must specify which mapping you want to use before moving between them. The author is teaching math, not engineering, and therefore must make this distinction.
Answers.UserDisplayName: Eponymous
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-30T18:15:05.283Z
Answers.Text: @Eponymous Both of them describes a vector space of dimension $nm$. In this case, multiple automorphisms exist for this object, two of which you gave as example. An automorphism always exists for any object (the identity mapping), and that is what the OP wanted to show.
Answers.UserDisplayName: PeaBrane
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-28T21:52:52.113Z
Answers.Text: @Eponymous "the author is teaching maths, not engineering" absolute savage hahahaha
Answers.UserDisplayName: David
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-31T08:16:26.473Z
Answers.LastActivityDate: 2022-08-31T08:16:26.473Z
Answers.UserDisplayName: PeaBrane
Body: <p>Consider the <a href="https://mml-book.github.io/book/mml-book.pdf#page=162" rel="nofollow noreferrer">following paragraph</a> from section <strong>5.4 Gradients fo Matrices</strong> of the chapter <strong>Vector Calculus</strong> from the textbook titled <a href="https://mml-book.github.io/book/mml-book.pdf" rel="nofollow noreferrer">Mathematics for Machine Learning</a> by <em>Marc Peter Deisenroth et al</em>.</p>
  <blockquote>
  <p>Since matrices represent linear mappings, <strong>we can exploit the fact that
  there is a vector-space isomorphism</strong> (linear, invertible mapping)
  between the space <span class="math-container">$\mathbb{R}^{m \times n}$</span> of <span class="math-container">$m \times n$</span> matrices
  and the space <span class="math-container">$\mathbb{R}^{mn}$</span> of mn vectors. <strong>Therefore, we can
  re-shape our matrices into vectors</strong> of lengths <span class="math-container">$mn$</span> and <span class="math-container">$pq$</span>,
  respectively. The gradient using these <span class="math-container">$mn$</span> vectors results in a
  Jacobian Matrices can be of size <span class="math-container">$mn \times pq$</span>. .... In practical
  applications, it is often desirable to re-shape the matrix into a
  vector and continue working with this Jacobian matrix: The chain
  rule... boils down to simple matrix multiplication, whereas in the
  case of a Jacobian tensor, we will need to pay more attention to what
  dimensions we need to sum out.</p>
  </blockquote>
  <p>What I understood from the paragraph is: There is always a <em>one-one mapping</em>(?) between <span class="math-container">$\mathbb{R}^{m \times n}$</span> and <span class="math-container">$\mathbb{R}^{mn}$</span>. So, we use this property to replace any element in <span class="math-container">$\mathbb{R}^{m \times n}$</span> (matrix) to an element in <span class="math-container">$\mathbb{R}^{mn}$</span>.</p>
  <p>I have doubt on how the property allows us to replace the matrix by vector without any discrepancies?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-03T11:35:38.377Z
Comments.Text: Can you please provide more details about your doubt? Why do you think that we cannot map $m \times n$ matrices to $m*n$ vectors without "any discrepancies"? What discrepancies or problems do you potentially see here? I am asking this question because I don't see any problem, but I am not thinking about all problems now. So, maybe, there's a problem or this has side-effects/consequences for certain operations.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-03T22:24:46.050Z
Comments.Text: @nbro The paragraph is saying vector-space isomorphism as the `only requisite` for reshaping. So, I asked with the intention of how it can assure compatibility with all tasks.
Comments.UserDisplayName: hanugm
Tags: <machine-learning><calculus><derivative><vector-space>
Title: How the vector-space isomorphism between $\mathbb{R}^{m \times n}$ and $\mathbb{R}^{mn}$ guarantees reshaping matrices to vectors?
UserDisplayName: hanugm
ThreadText: # How the vector-space isomorphism between $\mathbb{R}^{m \times n}$ and $\mathbb{R}^{mn}$ guarantees reshaping matrices to vectors?
  
  (Asked by: username_0 on 2021-11-30)
  
  Consider the [following paragraph](https://mml-book.github.io/book/mml-book.pdf#page=162) from section **5.4 Gradients fo Matrices** of the chapter **Vector Calculus** from the textbook titled [Mathematics for Machine Learning](https://mml-book.github.io/book/mml-book.pdf) by *Marc Peter Deisenroth et al*.
  
  > Since matrices represent linear mappings, **we can exploit the fact that
  > there is a vector-space isomorphism** (linear, invertible mapping)
  > between the space $\mathbb{R}^{m \times n}$ of $m \times n$ matrices
  > and the space $\mathbb{R}^{mn}$ of mn vectors. **Therefore, we can
  > re-shape our matrices into vectors** of lengths $mn$ and $pq$,
  > respectively. The gradient using these $mn$ vectors results in a
  > Jacobian Matrices can be of size $mn \times pq$. .... In practical
  > applications, it is often desirable to re-shape the matrix into a
  > vector and continue working with this Jacobian matrix: The chain
  > rule... boils down to simple matrix multiplication, whereas in the
  > case of a Jacobian tensor, we will need to pay more attention to what
  > dimensions we need to sum out.
  
  What I understood from the paragraph is: There is always a *one-one mapping*(?) between $\mathbb{R}^{m \times n}$ and $\mathbb{R}^{mn}$. So, we use this property to replace any element in $\mathbb{R}^{m \times n}$ (matrix) to an element in $\mathbb{R}^{mn}$.
  
  I have doubt on how the property allows us to replace the matrix by vector without any discrepancies?
  
  **Tags:** machine-learning, calculus, derivative, vector-space
  > **Comments:**
  > **username_1**: Can you please provide more details about your doubt? Why do you think that we cannot map $m \times n$ matrices to $m*n$ vectors without "any discrepancies"? What discrepancies or problems do you potentially see here? I am asking this question because I don't see any problem, but I am not thinking about all problems now. So, maybe, there's a problem or this has side-effects/consequences for certain operations.  
  > **username_0**: @nbro The paragraph is saying vector-space isomorphism as the `only requisite` for reshaping. So, I asked with the intention of how it can assure compatibility with all tasks.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2021-12-03. Score: 0
  
  An isomorphism $T$ between vector spaces $V$ and $W$ over the same field $K$ (for example, $K = \mathbb{R}$) is defined as a [**bijective**](http://www2.clarku.edu/%7Edjoyce/ma130/isomorphism.pdf) (i.e. 1-to-1 and onto, which makes a bijection an invertible function) transformation (a function) that preserves the 2 main properties
  
  1. scalar multiplication: $T(c\mathbf {u} )=cT(\mathbf {u} )$, where $c \in K$ and $\mathbf {u} \in V$
  2. vector addition: $T(\mathbf {u} +\mathbf {v} )=T(\mathbf {u} )+T(\mathbf {v})$, where $\mathbf {u}, \mathbf {v} \in V$
  
  Linear maps preserve these 2 properties, but not all linear maps are bijective, so not all linear maps are isomorphisms. In fact, linear maps that are bijective are called linear isomorphisms. You can denote an isomorphism between $V$ and $W$ as follows (to emphasize that it's not just a linear map): $T: V {\overset {\sim }{\to }} W$.
  
  The fact that an isomorphism is invertible means that there exists a function $T^{-1}$ that transforms any element of $W$ to any element of $V$.
  
  So, if there's an isomorphism $T$ between $V = \mathbb{R}^{n \times m}$ and $W = \mathbb{R}^{nm}$ (note that I am not proving it here, but it's probably not complicated to prove), then it implies that, for each $\mathbf {v}\_W \in \mathbb{R}^{nm}$, you can use $T^{-1}: \mathbb{R}^{nm} {\overset {\sim }{\to }} \mathbb{R}^{n \times m}$ to retrieve the corresponding vector $\mathbf {v}\_V \in \mathbb{R}^{n \times m}$ (not that, in the context of vector spaces, matrices are also vectors, which is just the name used to denote objects in *vector* spaces).
  
  This should prove why, if reshaping is an isomorphism, then you can perform operations first on $\mathbf {v}\_W$ and then go back to $\mathbf {v}\_V$.
  
  > **Comments:**
  > **username_1**: To prove reshaping is an isomorphism, you need to prove it's bijective and show that it preserves scalar and vector addition. This shouldn't be too difficult (I think, at least if you had one course in linear algebra), but if you have problems, let me know and I can try to help.  
  
  ---
  
  ### Answer by username_2 on 2022-08-31. Score: 0
  
  I think the textbook is making it unnecessarily complicated. $\mathbb{R}^{n\times m}$ and $\mathbb{R}^{nm}$ both represent the same vector space, a real vector space of dimension $nm$. (So clearly they are isomorphic because they are the same.)
  
  The interpretation of them being either a space of vectors or a space of matrices is an artificial distinction in this context.
  
  > **Comments:**
  > **username_3**: $\mathbb{R}^{n\times m}$ and $\mathbb{R}^{nm}$ are different spaces: matrices and vectors. Each real value in a matrix has a row and a column. Each value in a vector has just an index. There are two standard mappings between them: row-major and column-major, plus many uncommon mappings. Therefore they are not "the same" space (although they are isomorphic.) You must specify which mapping you want to use before moving between them. The author is teaching math, not engineering, and therefore must make this distinction.  
  > **username_2**: @Eponymous Both of them describes a vector space of dimension $nm$. In this case, multiple automorphisms exist for this object, two of which you gave as example. An automorphism always exists for any object (the identity mapping), and that is what the OP wanted to show.  
  > **username_4**: @Eponymous "the author is teaching maths, not engineering" absolute savage hahahaha  
  
  ---
  
  