Hi ptrblock,
I appreciate the tips. I think these would work for what I’m currently looking at (well, minus the two GPUs… ).
I would still like to look in to this deferred backprop approach, though. If for no other reason than that I’m curious what kind of performance penalties it would entail. If anyone has any tips on where the backprop graph’s states are stored, I’d really appreciate it.