Body: <p>Hi I have a very simple model and I'm trying to learn the math of it.</p>
  <p>Basically, I have an input matrix X m x n. An output matrix Y m x n is formed from some convolution H. The figure of merit is calculated for Y through the adjoint method (this is for electromagnetics). It produces a Gradient map G of size m x n that gives the derivative of the FOM with respect to teach element of Y.</p>
  <p>Now, I have the derivative of each element of Y with respect to G. This is a Jacobian of m x n x m x n I believe.</p>
  <p>What I want to know is how to obtain the derivative of FOM with respect to X with these two things, the Jacobian J and the gradient matrix G. This will give me the derivative of the FOM with respect to every element in X.</p>
  <p>Is it J x G or J dot G?</p>
  <p>Just to let you know, the reason for this is because I have the gradient map, and I have the Jacobian, but this is not part of a model that I can simply use backprop.</p>
  <p>Thanks</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-09T07:37:40.183Z
Comments.Text: It's just the chain rule. To get help, please give the mathematics for Y=f(X) in detail, and someone can show you. Right now it's not that clear how you have set up the convolution. Also are you sure you want gradient of X and not the gradient of H? It depends which one you want to change
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-09T12:46:43.217Z
Comments.Text: can you draw a high level block diagram of the forward process with the name of each element on top of it? once the forward process is clear, backpropagation should be straightforward
Comments.UserDisplayName: saleh
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-09T21:29:09.490Z
Comments.Text: I dont think the convolution matters. Lets just say its a linear filter. Literally just 
  kernel = torch.ones((1, 1, 3, 3)) / 9
  
  since its linear, J is just a sparse matrix that explains how the output pixels relate to the input pixels. Mind you, its independent of both the input and output (since the filter is the same every time.)
  
  I just want to know how we go from the first stage of the backprop (FOM vs output) to the second stage (FOM vs input) mathematically.
Comments.UserDisplayName: James Li
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-07-10T07:22:53.547Z
Comments.Text: The maths of the filter does matter. Not it's numbers, but some structure to X, Y and  H please, plus the math for the forward calculation, because someone can only show you the maths of the back propagation if they have that information
Comments.UserDisplayName: Neil Slater
Tags: <neural-networks><backpropagation><math><gradient>
Title: Backpropagation math question
UserDisplayName: James Li
ThreadText: # Backpropagation math question
  
  (Asked by: username_0 on 2024-07-09)
  
  Hi I have a very simple model and I'm trying to learn the math of it.
  
  Basically, I have an input matrix X m x n. An output matrix Y m x n is formed from some convolution H. The figure of merit is calculated for Y through the adjoint method (this is for electromagnetics). It produces a Gradient map G of size m x n that gives the derivative of the FOM with respect to teach element of Y.
  
  Now, I have the derivative of each element of Y with respect to G. This is a Jacobian of m x n x m x n I believe.
  
  What I want to know is how to obtain the derivative of FOM with respect to X with these two things, the Jacobian J and the gradient matrix G. This will give me the derivative of the FOM with respect to every element in X.
  
  Is it J x G or J dot G?
  
  Just to let you know, the reason for this is because I have the gradient map, and I have the Jacobian, but this is not part of a model that I can simply use backprop.
  
  Thanks
  
  **Tags:** neural-networks, backpropagation, math, gradient
  
  > **Comments:**
  > **username_1**: It's just the chain rule. To get help, please give the mathematics for Y=f(X) in detail, and someone can show you. Right now it's not that clear how you have set up the convolution. Also are you sure you want gradient of X and not the gradient of H? It depends which one you want to change  
  > **username_2**: can you draw a high level block diagram of the forward process with the name of each element on top of it? once the forward process is clear, backpropagation should be straightforward  
  > **username_0**: I dont think the convolution matters. Lets just say its a linear filter. Literally just  kernel = torch.ones((1, 1, 3, 3)) / 9  since its linear, J is just a sparse matrix that explains how the output pixels relate to the input pixels. Mind you, its independent of both the input and output (since the filter is the same every time.)  I just want to know how we go from the first stage of the backprop (FOM vs output) to the second stage (FOM vs input) mathematically.  
  > **username_1**: The maths of the filter does matter. Not it's numbers, but some structure to X, Y and  H please, plus the math for the forward calculation, because someone can only show you the maths of the back propagation if they have that information  
  
  ---
  
  ## 0 answers
  
  