Answers.Body: <p>The fact is you can always express an affine transformation as a linear transformation (more convenient because it is just a matrix/dot product).</p>
  <p>For instance, given an input <span class="math-container">$\textbf{x}=[x_1, ..., x_n]$</span>, some weights <span class="math-container">$\textbf{a} = [a_1, a_2, ..., a_n]$</span> and a bias <span class="math-container">$b \in \mathbb{R}$</span>, you can express the <strong>affine</strong> operation <span class="math-container">$y = \textbf{a}\cdot \textbf{x} + b$</span> as :</p>
  <p><span class="math-container">$y = \tilde{\textbf{a}} \cdot \tilde{\textbf{x}}$</span>, with <span class="math-container">$\tilde{\textbf{a}} = [a_1, ..., a_n, b]$</span> and <span class="math-container">$\tilde{\textbf{x}} = [x_1, ..., x_n, 1]$</span> (<strong>linear</strong> operation)</p>
  <p>When your affine transformation is a function <span class="math-container">$f:\mathbb{R}^p \rightarrow \mathbb{R}^q$</span> with <span class="math-container">$\textbf{y}=f(\textbf{x})=A\textbf{x} + \textbf{b}$</span>, you can use the same trick (by adding a column with the biases at the right end of the weight matrix <span class="math-container">$A$</span>), so you get: <span class="math-container">$\textbf{y}=\tilde{A}\tilde{\textbf{x}}$</span></p>
  <p>I found an example <a href="https://youtu.be/Q4GNLhRtZNc?t=398" rel="nofollow noreferrer">in this video</a>, where Andrew Ng uses this trick for a simple Linear Regression.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-18T16:40:31.460Z
Answers.LastActivityDate: 2021-07-19T10:43:19.273Z
Answers.LastEditDate: 2021-07-19T10:43:19.273Z
Answers.UserDisplayName: y-prudent
Answers.Body: <p>In <em>linear algebra</em>, a <a href="https://en.wikipedia.org/wiki/Linear_map" rel="noreferrer"><strong>linear transformation</strong></a> (aka linear map or linear transform) <span class="math-container">$f: \mathcal{V} \rightarrow \mathcal{W}$</span> is a function that satisfies the following two conditions</p>
  <ol>
  <li><span class="math-container">$f(u + v)=f(u)+f(v)$</span> (additivity)</li>
  <li><span class="math-container">$f(\alpha u) = \alpha f(u)$</span> (scalar multiplication),</li>
  </ol>
  <p>where</p>
  <ul>
  <li><span class="math-container">$u$</span> and <span class="math-container">$v$</span> vectors (i.e. elements of a <a href="https://en.wikipedia.org/wiki/Vector_space" rel="noreferrer">vector space</a>, which can also be <span class="math-container">$\mathbb{R}$</span> [<a href="https://proofwiki.org/wiki/Real_Numbers_form_Vector_Space" rel="noreferrer">proof</a>], some space of functions, etc.)</li>
  <li><span class="math-container">$\alpha$</span> is a scalar (e.g. which can be a real number, but not necessarily)</li>
  <li><span class="math-container">$\mathcal{V}$</span> and <span class="math-container">$\mathcal{W}$</span> are vector spaces (e.g. <span class="math-container">$\mathbb{R}$</span> or <span class="math-container">$\mathbb{R}^2$</span>)</li>
  </ul>
  <p>So, any function that satisfies these two conditions is a linear transformation.</p>
  <p>In <em>Euclidean geometry</em>, <span class="math-container">$g(x) = ax + b$</span> is an <a href="https://en.wikipedia.org/wiki/Affine_transformation" rel="noreferrer"><strong>affine transformation</strong></a>, which is generally <strong>not</strong> a linear transformation as defined in linear algebra. You can easily show that affine transformations are not linear transformations. For example, let <span class="math-container">$a = 1$</span> and <span class="math-container">$b = 2$</span>, does <span class="math-container">$g$</span> satisfy the second condition above for any scalar <span class="math-container">$\alpha$</span>? No. For example, let <span class="math-container">$\alpha = 3$</span>, then <span class="math-container">$g(3x = y) = y + 2 = 3x + 2 \neq 3 g(x) = 3 (x + 2) = 3x + 6$</span>.</p>
  <p>However, in the context of neural networks, when people use the adjective &quot;linear&quot; they are often referring to a line. For example, in linear regression, you can have a bias (the <span class="math-container">$b$</span> in the affine transformation <span class="math-container">$g$</span> above), which would make the function not a linear transformation, but we still call it linear regression because we fit a <strong>line</strong> (hence the name <strong>line</strong>ar regression) to the data.</p>
  <p>So, no, an affine transformation is not a linear transformation as defined in linear algebra, but all linear transformations are affine. However, in machine learning, people often use the adjective linear to refer to straight-line models, which are generally represented by functions that are affine transformations. In <a href="https://ai.stackexchange.com/a/20380/2444">this answer</a>, I also talk about this issue.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-17T02:30:10.577Z
Answers.Text: Ha, affine does not satisfying first property also.
Answers.UserDisplayName: hanugm
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-17T00:24:18.363Z
Answers.LastActivityDate: 2021-07-17T00:37:03.773Z
Answers.LastEditDate: 2021-07-17T00:37:03.773Z
Answers.UserDisplayName: nbro
Body: <p>Consider the following statements from <a href="https://pytorch.org/docs/stable/notes/modules.html" rel="nofollow noreferrer">A Simple Custom Module</a> of PyTorch's documentation</p>
  <blockquote>
  <p>To get started, let’s look at a simpler, custom version of PyTorch’s
  Linear module. This module applies an <strong>affine transformation</strong> to its
  input.</p>
  </blockquote>
  <p>Since the paragraph is saying <em>PyTorch’s <strong>Linear</strong> module</em>, I am guessing that affine transformation is nothing but linear transformation.</p>
  <p>Suppose <span class="math-container">$x = [x_1, x_2, x_3,\cdots,x_n]$</span> be an input, then the linear transformation on <span class="math-container">$x$</span> can be <span class="math-container">$a.x+b$</span>, where <span class="math-container">$a$</span> and <span class="math-container">$b$</span> are <span class="math-container">$n-$</span> dimensional vectors of real numbers. And dot(<span class="math-container">$.$</span>) stands for dot product.</p>
  <p>Is affine transformation same as the linear transformation? If yes, then why the name <strong>affine</strong> is used? Does it cover something more or less than linear transformation?</p>
  
Tags: <neural-networks><terminology><pytorch><linear-algebra>
Title: Is there any difference between affine transformation and linear transformation?
UserDisplayName: hanugm
ThreadText: # Is there any difference between affine transformation and linear transformation?
  
  (Asked by: username_0 on 2021-07-16)
  
  Consider the following statements from [A Simple Custom Module](https://pytorch.org/docs/stable/notes/modules.html) of PyTorch's documentation
  
  > To get started, let’s look at a simpler, custom version of PyTorch’s
  > Linear module. This module applies an **affine transformation** to its
  > input.
  
  Since the paragraph is saying *PyTorch’s **Linear** module*, I am guessing that affine transformation is nothing but linear transformation.
  
  Suppose $x = [x\_1, x\_2, x\_3,\cdots,x\_n]$ be an input, then the linear transformation on $x$ can be $a.x+b$, where $a$ and $b$ are $n-$ dimensional vectors of real numbers. And dot($.$) stands for dot product.
  
  Is affine transformation same as the linear transformation? If yes, then why the name **affine** is used? Does it cover something more or less than linear transformation?
  
  **Tags:** neural-networks, terminology, pytorch, linear-algebra
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2021-07-17. Score: 7
  
  In *linear algebra*, a [**linear transformation**](https://en.wikipedia.org/wiki/Linear_map) (aka linear map or linear transform) $f: \mathcal{V} \rightarrow \mathcal{W}$ is a function that satisfies the following two conditions
  
  1. $f(u + v)=f(u)+f(v)$ (additivity)
  2. $f(\alpha u) = \alpha f(u)$ (scalar multiplication),
  
  where
  
  * $u$ and $v$ vectors (i.e. elements of a [vector space](https://en.wikipedia.org/wiki/Vector_space), which can also be $\mathbb{R}$ [[proof](https://proofwiki.org/wiki/Real_Numbers_form_Vector_Space)], some space of functions, etc.)
  * $\alpha$ is a scalar (e.g. which can be a real number, but not necessarily)
  * $\mathcal{V}$ and $\mathcal{W}$ are vector spaces (e.g. $\mathbb{R}$ or $\mathbb{R}^2$)
  
  So, any function that satisfies these two conditions is a linear transformation.
  
  In *Euclidean geometry*, $g(x) = ax + b$ is an [**affine transformation**](https://en.wikipedia.org/wiki/Affine_transformation), which is generally **not** a linear transformation as defined in linear algebra. You can easily show that affine transformations are not linear transformations. For example, let $a = 1$ and $b = 2$, does $g$ satisfy the second condition above for any scalar $\alpha$? No. For example, let $\alpha = 3$, then $g(3x = y) = y + 2 = 3x + 2 \neq 3 g(x) = 3 (x + 2) = 3x + 6$.
  
  However, in the context of neural networks, when people use the adjective "linear" they are often referring to a line. For example, in linear regression, you can have a bias (the $b$ in the affine transformation $g$ above), which would make the function not a linear transformation, but we still call it linear regression because we fit a **line** (hence the name **line**ar regression) to the data.
  
  So, no, an affine transformation is not a linear transformation as defined in linear algebra, but all linear transformations are affine. However, in machine learning, people often use the adjective linear to refer to straight-line models, which are generally represented by functions that are affine transformations. In [this answer](https://ai.stackexchange.com/a/20380/2444), I also talk about this issue.
  
  > **Comments:**
  > **username_0**: Ha, affine does not satisfying first property also.  
  
  ---
  
  ### Answer by username_1 on 2021-07-18. Score: 4
  
  The fact is you can always express an affine transformation as a linear transformation (more convenient because it is just a matrix/dot product).
  
  For instance, given an input $\textbf{x}=[x\_1, ..., x\_n]$, some weights $\textbf{a} = [a\_1, a\_2, ..., a\_n]$ and a bias $b \in \mathbb{R}$, you can express the **affine** operation $y = \textbf{a}\cdot \textbf{x} + b$ as :
  
  $y = \tilde{\textbf{a}} \cdot \tilde{\textbf{x}}$, with $\tilde{\textbf{a}} = [a\_1, ..., a\_n, b]$ and $\tilde{\textbf{x}} = [x\_1, ..., x\_n, 1]$ (**linear** operation)
  
  When your affine transformation is a function $f:\mathbb{R}^p \rightarrow \mathbb{R}^q$ with $\textbf{y}=f(\textbf{x})=A\textbf{x} + \textbf{b}$, you can use the same trick (by adding a column with the biases at the right end of the weight matrix $A$), so you get: $\textbf{y}=\tilde{A}\tilde{\textbf{x}}$
  
  I found an example [in this video](https://youtu.be/Q4GNLhRtZNc?t=398), where Andrew Ng uses this trick for a simple Linear Regression.
  
  
  ---
  
  