If I have a BxCxHxW Variable which has a graph associated with it (It is the output of a CNN) and another similar Variable which is the output of another CNN. Now I want to replace a part of the first by second and pass the new Variable through a third network and then call backward on loss.
Right now, I run into an error because some Variables have been modified in-place. To fix that I am calling clone and then replacing in the third variable one by one, like this:
    def refine(old_rep, fine_rep, ys, xs):
        new_rep = old_rep.clone()
        for i in range(fine_rep.size(0)):
            for j in range(10):
                img_id = i // 10
                y = int(ys[i])
                x = int(xs[i])
               new_rep[img_id,:,y:y+1,x:x+1] = fine_rep[i]
        return new_rep

Is there a faster way to do the same?
Thanks