Thanks, should I be using retain_graph=True over create_graph=True?
My question which I cannot answer yet is why .backward() cannot backprop through the gradients? What exactly is it that makes it require retain_graph=True or create_graph=True?