Well, as no smarter person is replaying I will.
You can probably use register backward hooks. Each time a the hooked layer gets backwarded you will get an output.
The problem is it will be called as many times as gpus you have and I have not hints about how to deal with that.