We should have tried the checkpoint save. Just printing the model state_dict info was misleading. It seems we found that it is a bug. It was a bug … the actual model may have been smaller due to the way it was initialized. Sorry about that … will close this issue. Thanks!