Hi there,
It’s been a while, but basically the code was loading very large tensors into memory (I believe this was model-based RL, so very large trajectory data). What believe I did was create a scaled down version which used much smaller hyperparameters (e.g., rollout horizon) that would reduce the memory usage. This meant that I was able to get meaningful breakdowns in where time was being taken up.