GitHub is unfortunately down at the moment, but if I remember correctly, this issue could be solved by passing a dummy input with requires_grad=True to the checkpointed model?
PS: You could also check out the nightly binaries with native amp support 33