Detaching h2 earlier (e.g. right before i2) means gradients from h3.backward won’t flow through it, which I do want.

To make things less mysterious, this is a dynamic batching POC for RNNs: batch size of 2 (initially), and the three sequences are [a1, a2], [b1, b2, b3], [c1, c2] batched like this (each column is a timestep, first row is t labels):
0--1--2--3--4
|a1|a2|c1|c2|
|b1|b2|b3|

Sequence a gets backpropped at t = 2 (h2.backward), but graph can’t be pruned for t < 2 because b isn’t complete so non-zero gradients still need to flow there. b gets backpropped at t = 3, so now the graph can be pruned.

From what I’ve seen so far it looks like it’s not possible to modify an existing graph. Is there any way of creating a new graph, without redoing the forward operations?