Sure, if it’s hard for me that I’ve written it, it’s even harder for you since you are not familiar with it.
I didn’t know about torchviz and I had some problem using it on windows but here is the first graph:
first5604×6113 999 KB
I couldn’t update the second because it’s too big but it basically the second iteration with the first one stacked above it (since to backpropagate on acc loss all the previous iterations are needed).
Now, I don’t know if you can download it in full resolution but on the center-left of the image, you can see a one-element tensor i.e. the learning rate for that step.
In the graph at the second iteration, there are both the learning rate for the second step and, the one for the previous step. Basically, what I think is happening is that, at a given inner iteration, I’m not only updating the current learning rate but also all of the past ones.
Is there any way I can avoid this?