Hi! Really thanks for your concern! My stuff is a little complex, so I’ll try my best brief it up.
I am doing Neural Architecture Search specifically. The  MixedOp  here means an edge between two nodes, and it contains several candidate operations. During every iteration, I select one candidate operation for the forward process. The operation is specified by the variable  self.alpha_gate , and the active one has id  active_index[0] . And during the backward, I want to compute the gradients w.r.t the binary gates, i.e.  self.alpha_gate .
I copied the equation of computing the gradients from an open-source paper, so I think it is fine. I am only confused about the Runtime Error.