Sorry about that @soulitzer , This is my use case. Hope you can shed some light.
I am implementing a pipeline, consisting of 2 models M1 and M2. The pipeline is divided into 3 steps.
1. Train M1 on a dataset D1_train
a. y=model1(D1_train)
b. L1=(y-actual)*A
c. l1.backward()
d. delL1/delWo=(torch.autograd.grad(L1, model1.parameters(), create_graph=True, allow_unused=True)) # model1's weights -> Wo
e. params=[]
   out_params=[]
   for i in range(len(delL1/delWo)):
     if delL1/delWo[i]!=None:
       params.append(delL1/delWo[i])
       out_params.append(torch.ones_like(delL1_delWo[i]))
   del2_L1/delWo_delA=torch.autograd.grad(params, A, allow_unused=True, grad_outputs=out_params) #calculating gradients wrt A i.e del^2 L1/delWo delA
f. optimizer1.step()   #to update model1's weights

2. Train M2 on a dataset (D2) generated by trained M1
a. D2=model1(inp)
b. y=model2(D2)
c. L2=y-actual
d. L2.backward()
e. delL2/delW=(torch.autograd.grad(loss2, model2.parameters(), create_graph=True, allow_unused=True, only_inputs=True)) #model2's weights -> W
f. params=[]
   out_params=[]
   for i in range(len(delL2/delW)):
      if delL2/delW[i]!=None:
        params.append(delL2/delW[i])
        out_params.append(torch.ones_like(delL2/delW[i]))
    for param1 in model1.parameters():
      grad=torch.autograd.grad(params, param1, allow_unused=True, grad_outputs=out_params, retain_graph=True)[0]
      del2_L2/delW_delWo.append(grad)
g. optimizer2.step() #to update model2's weights

3. Train A by reducing validation loss of model2
a. y=model2(D1_val)
b. L3=y-actual
c. delL3_delW=torch.autograd.grad(L3, model2.parameters())
// Wnat to update A using the following gradient 
// delL3/delA = (delL3/delW) x (del2_L2/delW_delWo) x (del2_L1/delWo_delA) 
d. A.grads= delL3/delW x del2_L2/delW_delWo x del2_L1/delWo_delA -> how to calculate this? (essentially a chain rule -> delWo/delA x delW/delWo x delL3/delW)
e. optimizer3.step() #to update A

So I was able to calculate, del2_L2/delW_delWo,  del2_L1/delWo_delA  and delL3/delW. But the shapes of all three tensors are different.
del2_L2/delW_delWo and delL3/delW → length 512 and containing tensors of variable dimensions. (Eg: ((tensor of size 10x10), (tensor of size 10), …) → dummy sizes)
del2_L1/delWo_delA → tensor of size (1x5 )

Update:
I tried doing the dot product of the tensors in del2_L2/delW_delWo and delL3/delW and adding the
results to get a new gradient. Then I multiplied the obtained gradient by del2_L1/delWo_delA. Is this right? ref: neural network - Pytorch, what are the gradient arguments - Stack Overflow (second answer) It was mentioned in the answer, According to chain rule, in order to calculate gradient of loss w.r.t to a leaf node, we can compute derivative of loss w.r.t some intermediate variable, and gradient of intermediate variable w.r.t to the leaf variable, do a dot product and sum all these up. I used this approach as I wanted to implement chain rule manually.