I’m using pycharm in debug mode - which i believe uses dbg somewhere in there.
I finally got things to work. I found that having very long sequences ( 2000+ time-steps ) caused the issue. It may have been due to my loader. I didn’t inspect it in depth since I found a solution.
It doesn’t repeat itself on simple small models and I can’t really share my production architecture/training procedures.
if you sack a bunch of 1d convs so that you get a model in the 10’s of millions of parameters and try to run it on a small batch of a very long sequence you may get similar errors.
Thanks for getting back to me.