@ptrblck  Can we move a little further from this topic, say what if I want to get all intermediate results of all the operations in the graph during forward propagation. Because this is quite simple to do in tensorflow, you can

Load the graph;
Get all tensors from the graph;
e.g.
all_ops = graph.get_operations()
all_op_names = [op.name for op in all_ops]
all_tensors = [graph.get_tensor_by_name(’{}:0’.format(op_name)) for op_name in all_op_names]
For tf 1.15 which I currently use a lot, use output = session.run(all_tensors, feed_dict={d_input:input_images}) to get all output. Can we do the same thing here in PyTorch. The reason why I asked this, is I want to check all operations’ results running on two different platform, and check that the results are all the same. If some intermmediate result differs, I can spot that.
Another question is can we get all operations in a training graph including all the backprop related operations?
Much appreciated.