Hey,
I can’t reproduce on master or with the nightly builds:
[albandes ~/tmp/create_graph_debug] git pull
Updating 15a5a85..169926d
Fast-forward
 test.py | 21 ++++++++++++---------
 1 file changed, 12 insertions(+), 9 deletions(-)
[albandes@devvm138.atn0 ~/tmp/create_graph_debug] . ../../local/pytorch/3.6_debug_source_env/bin/activate
(3.6_debug_source_env) [albandes ~/tmp/create_graph_debug] python test.py
when create_graph is True
loss                     :15.955138471719597
/data/users/albandes/pytorch/3.6_debug_source/torch/functional.py:1242: UserWarning: torch.norm is deprecated and may be removed in a future PyTorch release. Use torch.linalg.norm instead.
  "torch.norm is deprecated and may be removed in a future PyTorch release. "
classifier grad norm     :394.79707122347384
bert embedding grad norm :19.880596727129774
When create_graph is False
loss                     :15.955138471719597
classifier grad norm     :394.79707122347384
bert embedding grad norm :19.880596727129774

But I can on 1.6:
(3.6_release_binary_env) [albandes@devvm138.atn0 ~/tmp/create_graph_debug] python test.py
when create_graph is True
loss                     :15.955138471719597
classifier grad norm     :394.7970712234737
bert embedding grad norm :23.12885296693177
When create_graph is False
loss                     :15.955138471719597
classifier grad norm     :394.7970712234737
bert embedding grad norm :19.88059672712989

Can you install the nightly build from here https://pytorch.org/get-started/locally/ 4 and confirm this is working fine for you?