Body: <p>I work with a few different automatic differentiation frameworks, including pytorch, Jax, and Flux in Julia. Periodically I run some code and I get errors about mutations or operations occurring &quot;in-place.&quot; These errors generally cause the program to fail. <em>My question is, what does this error mean--in the sense of the AD algorithm and not the specific programming framework?</em></p>
  <p>So I wanted to understand from a very low level, what the problem is and how to get around it. I believe most AD frameworks focus on reverse mode AD because it is more efficient for a large number of parameters.</p>
  <p>I am not sure how current AD algorithms are implemented. Older AD systems were based on Wengert lists, while more recent implementations use source-to-source transformations. So just to elaborate on my original equation, does in-place mutation create problems for both source-to-source AD packages and Wengert list/tape based AD packages?</p>
  <p>Here is my guess at the issue, though I cannot confirm it. It seems like the whole mutation issue has something to do with computing either the forward pass or the backward pass of the AD algorithm. So to compute the gradient, the current value of each variable needs to be known. If we change the value of a variable in the chain in-place, then we compute the wrong gradient at that variable, in that time-step. So that makes sense. But I am not sure if my understanding is correct here. So any clarification is appreciated.</p>
  <p>After that, how does one write code that avoids this kind of problem? I suppose this question can depend on the AD framework the user is using, but does any operation need to create a new variable instead of reusing existing variables? How smart are these frameworks in handling these types of mutations.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-12T10:59:25.633Z
Comments.Text: Please, ask only one question per post. Make sure that your question is on-topic (so programming questions are off-topic): https://ai.stackexchange.com/help/on-topic.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-12T16:02:43.590Z
Comments.Text: @nbro thanks for the response. The core question is "why do in-place mutations cause AD to fail?" I was trying to provide some context to the question meaning that I wanted to understand where in the reverse mode AD algorithm the problem occurs. I will update the question text to clarify my focus.
Comments.UserDisplayName: krishnab
Tags: <backpropagation><derivative>
Title: Why does "in-place" mutation cause automatic differentiation to fail, and how to write code to avoid this problem
UserDisplayName: krishnab
ThreadText: # Why does "in-place" mutation cause automatic differentiation to fail, and how to write code to avoid this problem
  
  (Asked by: username_0 on 2024-02-12)
  
  I work with a few different automatic differentiation frameworks, including pytorch, Jax, and Flux in Julia. Periodically I run some code and I get errors about mutations or operations occurring "in-place." These errors generally cause the program to fail. *My question is, what does this error mean--in the sense of the AD algorithm and not the specific programming framework?*
  
  So I wanted to understand from a very low level, what the problem is and how to get around it. I believe most AD frameworks focus on reverse mode AD because it is more efficient for a large number of parameters.
  
  I am not sure how current AD algorithms are implemented. Older AD systems were based on Wengert lists, while more recent implementations use source-to-source transformations. So just to elaborate on my original equation, does in-place mutation create problems for both source-to-source AD packages and Wengert list/tape based AD packages?
  
  Here is my guess at the issue, though I cannot confirm it. It seems like the whole mutation issue has something to do with computing either the forward pass or the backward pass of the AD algorithm. So to compute the gradient, the current value of each variable needs to be known. If we change the value of a variable in the chain in-place, then we compute the wrong gradient at that variable, in that time-step. So that makes sense. But I am not sure if my understanding is correct here. So any clarification is appreciated.
  
  After that, how does one write code that avoids this kind of problem? I suppose this question can depend on the AD framework the user is using, but does any operation need to create a new variable instead of reusing existing variables? How smart are these frameworks in handling these types of mutations.
  
  **Tags:** backpropagation, derivative
  
  > **Comments:**
  > **username_1**: Please, ask only one question per post. Make sure that your question is on-topic (so programming questions are off-topic): https://ai.stackexchange.com/help/on-topic.  
  > **username_0**: @username_1 thanks for the response. The core question is "why do in-place mutations cause AD to fail?" I was trying to provide some context to the question meaning that I wanted to understand where in the reverse mode AD algorithm the problem occurs. I will update the question text to clarify my focus.  
  
  ---
  
  ## 0 answers
  
  