Solved by Shisho_Sama in post #2 
              
              
                I guess your issue is your latest_input interfers with the current graph as the error suggest so, just do self.latest_input.clone().detach() so your only providing the data needed.