<p>Yes, typically, a fully connected layer is an affine transformation, which can or not be followed by a non-linear activation function, but, in many (if not most) cases, it's followed by a non-linearity, such as ReLU, sigmoid, or tanh (<a href="https://ai.stackexchange.com/a/31722/2444">an exception is when you do regression</a>), which is what makes the neural network be able to <a href="https://ai.stackexchange.com/a/13319/2444">approximate non-linear/complicated functions</a>.</p>

CC BY-SA 4.0
2021-09-20T00:20:23.567Z
2021-09-20T00:20:23.567Z
nbro
<p>We generally encounter the following statement several times</p>
<blockquote>
<p>The input vector is first fed into a <strong>fully connected layer</strong>......</p>
</blockquote>
<p>Since linear activation functions, such as identity function, can so considered as an activation functions, a fully connected layer can be considered just as an Affine transformation if the fully connected layer uses linear activation function.</p>
<p>So, in theory, a fully connected layer can refer to the following</p>
<ol>
<li>Just an affine transformation</li>
<li>Affine transformation followed by a nonlinear activation function</li>
</ol>
<p>Do authors generally choose to use &quot;fully connected layer&quot; for case 2 only or for both cases 1 and 2?</p>

CC BY-SA 4.0
2021-09-16T08:06:29.373Z
Affine transformations have this property that multiple affine transformations are equivalent to one affine transformation, so you gain nothing by stacking such layers.
Criticize SE actions means ban
CC BY-SA 4.0
2021-09-16T08:23:59.830Z
@user253751 consider a single one.
hanugm
CC BY-SA 4.0
2021-09-16T20:22:44.087Z
According to my experience, it refers to the 2nd case. When you're talking of neural networks, it's implicitly assumed you use a non-linear activation function at the end of each layer.
SpiderRico
<neural-networks><terminology>
Do authors generally use fully connected layer instead of affine transformation?
hanugm
# Do authors generally use fully connected layer instead of affine transformation?

(Asked by: username_0 on 2021-09-16)

We generally encounter the following statement several times

> The input vector is first fed into a **fully connected layer**......

Since linear activation functions, such as identity function, can so considered as an activation functions, a fully connected layer can be considered just as an Affine transformation if the fully connected layer uses linear activation function.

So, in theory, a fully connected layer can refer to the following

1. Just an affine transformation
2. Affine transformation followed by a nonlinear activation function

Do authors generally choose to use "fully connected layer" for case 2 only or for both cases 1 and 2?

**Tags:** neural-networks, terminology

> **Comments:**
> **username_1**: Affine transformations have this property that multiple affine transformations are equivalent to one affine transformation, so you gain nothing by stacking such layers.  
> **username_0**: @user253751 consider a single one.  
> **username_2**: According to my experience, it refers to the 2nd case. When you're talking of neural networks, it's implicitly assumed you use a non-linear activation function at the end of each layer.  

---

## 1 answers

### Answer by username_3 on 2021-09-20. Score: 1 | Accepted answer

Yes, typically, a fully connected layer is an affine transformation, which can or not be followed by a non-linear activation function, but, in many (if not most) cases, it's followed by a non-linearity, such as ReLU, sigmoid, or tanh ([an exception is when you do regression](https://ai.stackexchange.com/a/31722/2444)), which is what makes the neural network be able to [approximate non-linear/complicated functions](https://ai.stackexchange.com/a/13319/2444).


---

