Answers.Body: <p>ELU does not suffer from dying neurons issue, unlike ReLU. While ELU can help you to achieve better accuracy, it is slower than ReLU because of its non-linearity in its negative range.</p>
  <p>Choosing the right activation function totally depends on the situation but you need to also consider other similar types of activation functions such as leaky ReLU.</p>
  <p>Check this <a href="https://mlfromscratch.com/activation-functions-explained/#/" rel="nofollow noreferrer">link</a> out. It could be useful.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-27T14:37:26.807Z
Answers.LastActivityDate: 2021-12-22T14:45:07.567Z
Answers.LastEditDate: 2021-12-22T14:45:07.567Z
Answers.UserDisplayName: pedrum
Answers.Body: <p>The answer above makes some great comparisons/trade-offs. To help address the non-linearity issue with eLU units that the previous answer brings up, you can also use Leaky-ReLU units, which are linear in both the positive and negative range, and piecewise linear across the whole real domain.</p>
  <p>Please see the link <a href="https://qr.ae/pNsatH" rel="nofollow noreferrer">here</a> for more details.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-29T20:40:33.910Z
Answers.Text: Note that the answer you mention already mentions leaky ReLU.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-29T20:19:06.420Z
Answers.LastActivityDate: 2021-12-24T00:08:18.647Z
Answers.LastEditDate: 2021-12-24T00:08:18.647Z
Answers.UserDisplayName: Ryan S
Body: <p>I always use RELUs actication functions when I need to and I understand limitations of ELUs. So in what situation do I need to consider ELUs over RELUs?</p>
  
Tags: <neural-networks><deep-learning><activation-functions>
Title: In what situations ELUs should be used instead of RELUs?
UserDisplayName: McQubit
ThreadText: # In what situations ELUs should be used instead of RELUs?
  
  (Asked by: username_0 on 2020-07-27)
  
  I always use RELUs actication functions when I need to and I understand limitations of ELUs. So in what situation do I need to consider ELUs over RELUs?
  
  **Tags:** neural-networks, deep-learning, activation-functions
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2020-07-27. Score: 2 | Accepted answer
  
  ELU does not suffer from dying neurons issue, unlike ReLU. While ELU can help you to achieve better accuracy, it is slower than ReLU because of its non-linearity in its negative range.
  
  Choosing the right activation function totally depends on the situation but you need to also consider other similar types of activation functions such as leaky ReLU.
  
  Check this [link](https://mlfromscratch.com/activation-functions-explained/#/) out. It could be useful.
  
  
  ---
  
  ### Answer by username_2 on 2020-07-29. Score: 0
  
  The answer above makes some great comparisons/trade-offs. To help address the non-linearity issue with eLU units that the previous answer brings up, you can also use Leaky-ReLU units, which are linear in both the positive and negative range, and piecewise linear across the whole real domain.
  
  Please see the link [here](https://qr.ae/pNsatH) for more details.
  
  > **Comments:**
  > **username_3**: Note that the answer you mention already mentions leaky ReLU.  
  
  ---
  
  