<p>Have a look at these graphics showing popular linear units (image taken from <a href="https://arxiv.org/pdf/1511.07289v5.pdf" rel="noreferrer">Clevert et al. 2016</a>):
<a href="https://i.sstatic.net/gQVus.png" rel="noreferrer"><img src="https://i.sstatic.net/gQVus.png" alt="enter image description here" /></a></p>
<p>You can see that these functions are linear functions for <span class="math-container">$x &gt; 0$</span>, that's why they are called <em>Linear Units</em>.</p>
<p>For example, the ELU is defined as</p>
<p><span class="math-container">$$ ELU(x) = \begin{cases} 
x &amp;\text{if } x &gt; 0\\
\alpha (\exp(x)-1) &amp; \text{if } x \leq 0.
\end{cases} $$</span>
These functions introduce the nonlinearity around zero, each in its own way, which can be used for different problems.</p>

CC BY-SA 4.0
2021-07-29T13:27:07.627Z
2021-07-29T13:27:07.627Z
e.Fro
<p>Activation functions, in neural networks, are used to introduce non-linearity. Many activation functions that are used in neural networks have the term &quot;Linear Unit&quot; in their full form. &quot;Linear unit&quot; can be abbreviated as <strong>LU</strong>.</p>
<p>For example, consider some activation functions</p>
<blockquote>
<p>ELU  - Exponential Linear Unit</p>
<p>ReLU - Rectified Linear Unit</p>
<p>................................................</p>
</blockquote>
<p>Why does the function name contain the term &quot;Linear Unit&quot;? What is meant by <strong>Linear Unit</strong> here? Is it saying anything about the nature of function under consideration?</p>

<neural-networks><deep-learning><terminology><activation-functions>
What does "linear unit" mean in the names of activation functions?
hanugm
# What does "linear unit" mean in the names of activation functions?

(Asked by: username_0 on 2021-07-29)

Activation functions, in neural networks, are used to introduce non-linearity. Many activation functions that are used in neural networks have the term "Linear Unit" in their full form. "Linear unit" can be abbreviated as **LU**.

For example, consider some activation functions

> ELU - Exponential Linear Unit
>
> ReLU - Rectified Linear Unit
>
> ................................................

Why does the function name contain the term "Linear Unit"? What is meant by **Linear Unit** here? Is it saying anything about the nature of function under consideration?

**Tags:** neural-networks, deep-learning, terminology, activation-functions


---

## 1 answers

### Answer by username_1 on 2021-07-29. Score: 6 | Accepted answer

Have a look at these graphics showing popular linear units (image taken from [Clevert et al. 2016](https://arxiv.org/pdf/1511.07289v5.pdf)):
[![enter image description here](https://i.sstatic.net/gQVus.png)](https://i.sstatic.net/gQVus.png)

You can see that these functions are linear functions for $x > 0$, that's why they are called *Linear Units*.

For example, the ELU is defined as

$$ ELU(x) = \begin{cases}
x &\text{if } x > 0\\
\alpha (\exp(x)-1) & \text{if } x \leq 0.
\end{cases} $$
These functions introduce the nonlinearity around zero, each in its own way, which can be used for different problems.


---

