Is there any way of debugging the backwards pass? If I try and do it through PyCharm for example, I get to Variable._execution_engine.run_backward before I can no longer step through it – I assume that this part is in C? Is there any way I can look at the operations mentioned in the profiler specifically?
I’m also wondering if the time taken in the backwards pass is simply due to how many operations take place; running the profiler on only the backwards pass gives over 300,000 operations in the profiler. Printing the contents of the profiler alone takes ages…