Hi @Paralysis
You can register hooks to peek into the intermediate computations.   There’s an example on the pytorch docs, but I’ve pasted the example w/ some slight mods into this notebook gist: https://gist.github.com/braingineer/9b8205b16c6c2fa50a2afd85a5448742 101
You will have to write a function to recurse onto modules with submodules in order to register all submodules, but that should be pretty straightforward.