So I don’t know how to answer most of your questions and I haven’t looked in a while relative to the rate of change. Apparently, something is not quite there yet for the control flow in torch TVM.
For the fusion in the fusion pass, blocks is not currently handled, and I am ignorant of whether the obstacle torch TVM hit was with the graph rearrangement itself or something else.
to the best of my knowledge we don’t currently fuse inplace ops, so I would expect adventure when you try that. Maybe that is part of the problem you are seeing.
I don’t know any examples beyond the obvious users. In theory, I would expect the optimizations removing ifs with constant true/false result to give some idea of what it would take to move ops with blocks into a subgraph.
I’ll be very interested to hear from your progress.
Best regards
Thomas