Hi @James_Reed ,
Thanks for the quick response!
That makes sense — things seem to have worked out for my specific instance of the problem I was facing such that ShapeProp worked well enough, but for the general case one would definitely need symbolic shape tracing, particularly for keeping track of which dimensions of intermediate results are affected by the dimensions of the inputs.
(So its not really an issue of the efficiency of shape propagation with real Tensors — it’s a more fundamental limitation, in general.)
I’m sure that general symbolic shape analysis for a library the size of PyTorch would be a very painful project of dubious general value. What might be interesting in the future is providing a common repository of symbolic shape formulas somewhere in fx that people can contribute back to over time — basically a small but growing dict of rules an an associated example of a SymbolicShapeProp class. The biggest challenge, as far as I can tell, would be a symbolic version of broadcasting. Once you know how two shapes broadcast, most other shape transformations that I can think of follow simple rules. (Einsum or matmul, for example.)
Anyway, that’s not a feature request, just some thoughts. Really appreciate the work you’ve all been doing on fx — right before it was released and I learned about it I was writing my own very ad-hoc limited version of it, and having this has really simplified things.
Best,
A