Answers.Body: <p>I think it is by analogy with an electrical rectifier. A rectifier allows current to flow in one direction but blocks current in the other direction. Or if you prefer it allows voltage in one polarity but won't allow a voltage in the opposite polarity. Which is also what a ReLU does.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-28T12:03:05.260Z
Answers.LastActivityDate: 2022-06-28T12:03:05.260Z
Answers.UserDisplayName: Criticize SE actions means ban
Body: <p>ReLU stands for Rectified Linear Unit. Linear Unit, I understand, since the function is piecewise linear. But what does rectified mean?</p>
  <p>I looked up the definition and it said:</p>
  <blockquote>
  <p>denoting an electric current that has been converted from alternating current to direct current</p>
  </blockquote>
  <p>But I don't quite see how this can connect to ReLU function. While it does have a bend at <span class="math-container">$x = 0$</span> indicating a change in direction, not sure how I can work this into the idea of current directions.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-28T08:10:30.770Z
Comments.Text: I always linked it to engineering, a rectifier converts alternating current into direct current. As ReLU gets rid of the negative side of the linear function I think its fitting. Not sure though if thats also what the authors had in mind.
Comments.UserDisplayName: Chillston
Tags: <deep-learning><terminology><activation-functions><relu>
Title: Where does the "rectified" in ReLU come from?
UserDisplayName: a6623
ThreadText: # Where does the "rectified" in ReLU come from?
  
  (Asked by: username_0 on 2022-06-28)
  
  ReLU stands for Rectified Linear Unit. Linear Unit, I understand, since the function is piecewise linear. But what does rectified mean?
  
  I looked up the definition and it said:
  
  > denoting an electric current that has been converted from alternating current to direct current
  
  But I don't quite see how this can connect to ReLU function. While it does have a bend at $x = 0$ indicating a change in direction, not sure how I can work this into the idea of current directions.
  
  **Tags:** deep-learning, terminology, activation-functions, relu
  > **Comments:**
  > **username_1**: I always linked it to engineering, a rectifier converts alternating current into direct current. As ReLU gets rid of the negative side of the linear function I think its fitting. Not sure though if thats also what the authors had in mind.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-06-28. Score: 1 | Accepted answer
  
  I think it is by analogy with an electrical rectifier. A rectifier allows current to flow in one direction but blocks current in the other direction. Or if you prefer it allows voltage in one polarity but won't allow a voltage in the opposite polarity. Which is also what a ReLU does.
  
  
  ---
  
  