Also anticipating possible questions
I’ve introduced changes to fuse_modules.py 4
I propose that it is needed to preserve pre and post hooks on fused modules, where it possible. While it hard to define how to preserve hooks for second and third module in sequence (because input data changes and three modules are collapsing into atomic one). But we can easily preserve pre_forward hook of base module.
What cases can we process also?