Hey thanks a lot for running the code !
But I ran the code on cpu and made the changes you asked me to make yet I got different results based on create_graph 
I also fixed the random seed torch.manual_seed(0)
when create_graph is True
loss                     :15.955138471719597
classifier grad norm     :394.79707122347384
bert embedding grad norm :23.128852966924228
When create_graph is False
loss                     :15.955138471719597
classifier grad norm     :394.79707122347384
bert embedding grad norm :19.880596727123276

I have pushed my cpu runable version of my code in the git repo, could you run that please.
Could the difference be due to different version of pytorch ?
torch related libraries that I am using: torch==1.6.0, torchfile==0.1.0, torchvision==0.7.0
Thanks a lot  for spending your time on this !!