Detaching Variables
Something I noticed in transitioning from Pretrained-BERT to Transformers is that for the purposes of using BERT as a feature extractor/probing the pretrained representations, I need to detach variables whereas I previously didn't. I am not sure if this is noted somewhere (I didn't see it in the section in the docs about transitioning) but found it be highly relevant to prevent unnecessary memory usage. 
Maybe you are not using `with torch.grad()` when calling the model for inference?

I've added that in the readme example (it used to be mentioned there indeed).<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
