I’m using pytorch for some optimization problems and it has been a nice tool. Unfortunately I managed to create a script that produces results that I want, but for some reason it is getting slower with every iteration. I tried to reduce the code to a minimal implementation, but so far I have not managed to pinpoint the exact operation that causes the problem.
Here is the minimized script with timing code included: https://pastebin.com/2pvNUQUg 43
Could someone please verify the results (problem presents both with CPU and GPU). I have not noticed a change in memory usage. As can be seen, every 100 iteration takes around 0.5% more time to complete:
$ python3 pytorch_slowdown.py
3.5418 +254.2%
3.5648 +0.7%
3.5748 +0.3%
3.5940 +0.5%
3.6103 +0.5%
3.6276 +0.5%
3.6512 +0.7%
3.6621 +0.3%
3.6814 +0.5%