Hi @anna_l !
Thanks for your question and for taking interest in opacus.
I’d need some more info to be able to help, as I wasn’t able to reproduce the issue in my setup.

Can you please share which versions of transformers and opacus are you using?
Does the error happen on the first training iteration or later?

To comment on some of the discussion points above:


virtual_step() is a method defined in PrivacyEngine in opacus. It a way to simulate large batches without heavy memory footprint.
In our tutorial we indeed freeze some layers, as correctly pointed out. However, the error above lists trainable layers as not having gradients, which is not what should happen. (e.g. bert.encoder.layer.11 is bert.encoder.layer[-1])