@albanD  Thank you for helping me out, would you mind if I share a github link with the minimal example ?
I am using a model where the first layer is the bert embedding layer, and its attached to a classifier. The gradients for the classifier  however match with and without create_graph