Answers.Body: <p>Not sure wether this question is suitable here or should rather be on the math SE but here goes.</p>
  <p>First restrict to two arguments, the general case is similar. Let <span class="math-container">$f(x,y):=max(x,y)$</span>. This is a function defined on the entire real plane. When restricted to the region of the plane where <span class="math-container">$x&lt;y$</span> then in this region <span class="math-container">$f(x,y)=y$</span> which is clearly differentiable. Similarly in the region where <span class="math-container">$x&gt;y$</span> we have <span class="math-container">$f(x,y)=x$</span> which is also differentiable.</p>
  <p>However on the line where <span class="math-container">$x=y$</span> this function is continuous but not differentiable. To see this consider for example <span class="math-container">$max(t, 1-t)$</span> for a real value <span class="math-container">$t$</span>. This is continuous but has a kink at <span class="math-container">$t=1/2$</span> so is not differentiable at this point. This is exactly where it crosses the <span class="math-container">$x=y$</span> line.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-29T20:25:23.957Z
Answers.Text: Do you know how autodiff libraries handle the case $x=y$?
Answers.UserDisplayName: Antonios Sarikas
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-25T06:37:53.657Z
Answers.LastActivityDate: 2023-05-25T06:37:53.657Z
Answers.UserDisplayName: quarague
Body: <p>I recently came across an <a href="https://stackoverflow.com/a/63856493/7339624">answer</a> on StackOverflow that mentioned the <code>max</code> function being differentiable with respect to its values.</p>
  <p>From my current understanding of mathematics, I'm struggling to comprehend how this is possible.</p>
  <p>Could someone help explain how we can define the derivative of the max function in the case of multiple arguments as <code>max(a1, a2, ..., an)</code>?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-05-26T08:11:23.097Z
Comments.Text: Are you asking how the derivative of a function of multiple arguments is defined?
Comments.UserDisplayName: Plop
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-05-26T17:58:41.683Z
Comments.Text: @Plop No. Actually, someone changed the original title. The tile is not mine.
Comments.UserDisplayName: Peyman
Tags: <neural-networks><deep-learning><backpropagation><derivative><max-pooling>
Title: How is the max function differentiable wrt multiple arguments?
UserDisplayName: Peyman
ThreadText: # How is the max function differentiable wrt multiple arguments?
  
  (Asked by: username_0 on 2023-05-25)
  
  I recently came across an [answer](https://stackoverflow.com/a/63856493/7339624) on StackOverflow that mentioned the `max` function being differentiable with respect to its values.
  
  From my current understanding of mathematics, I'm struggling to comprehend how this is possible.
  
  Could someone help explain how we can define the derivative of the max function in the case of multiple arguments as `max(a1, a2, ..., an)`?
  
  **Tags:** neural-networks, deep-learning, backpropagation, derivative, max-pooling
  > **Comments:**
  > **username_1**: Are you asking how the derivative of a function of multiple arguments is defined?  
  > **username_0**: @Plop No. Actually, someone changed the original title. The tile is not mine.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-05-25. Score: 7 | Accepted answer
  
  Not sure wether this question is suitable here or should rather be on the math SE but here goes.
  
  First restrict to two arguments, the general case is similar. Let $f(x,y):=max(x,y)$. This is a function defined on the entire real plane. When restricted to the region of the plane where $x then in this region $f(x,y)=y$ which is clearly differentiable. Similarly in the region where $x>y$ we have $f(x,y)=x$ which is also differentiable.
  
  However on the line where $x=y$ this function is continuous but not differentiable. To see this consider for example $max(t, 1-t)$ for a real value $t$. This is continuous but has a kink at $t=1/2$ so is not differentiable at this point. This is exactly where it crosses the $x=y$ line.
  
  > **Comments:**
  > **username_3**: Do you know how autodiff libraries handle the case $x=y$?  
  
  ---
  
  