CPU Memory Leak when using RoBERTa for just word vector representation
Hi,
I do not use model for training or fine tuning. I just want to give strings and take their representations. My dataset is Robust04 (about 2G) and max length of each document is truncated to 1024 tokens. So, I break each documents to pieces of 64 tokens and represent it and then concatenate the representation of each 64 tokens pieces to get word vector representation of document with length of 1024 (1024*768). I use 64GB CPU RAM but it crashed after about 40% of documents represented. The used code is in the following:
`tokenizer = AutoTokenizer.from_pretrained('roberta-base')`
`model = AutoModel.from_pretrained('roberta-base')`
`cl_text = tokenizer.encode("I am using RoBERTa")`
`piece = tokenizer.decode(cl_text)`
`piece = tokenizer(piece, return_tensors="pt", max_length=le, pad_to_max_length=True, truncation=True,
					   return_attention_mask=True, return_token_type_ids=True, add_special_tokens=False)`
`piece = model(**piece)`
`piece = piece.last_hidden_state`
`piece.detach()`

Would you please guide me?

Thanks in advance,
Regards
Hi @ZahraGithub, 
1) Please use `model.eval()` to reduce your memory consumption.
2) From what I understood, you're processing 64 tokens in one go and total tokens are 1024. This means you'll get vector representation of (16,768) for each document right? How are you storing these representations? You won't be able to load all of them in your RAM hence that CPU memory leak. Try saving these representations to disk for each (that can vary) document to avoid memory leak.<|||||>Hi @bhavitvyamalik

1. I did it but again the memory consumption is high.
2. I can not understand what is the amount of 16768?<|||||>For 2, I think what's happening is you're storing all your representations in a list or something. You should load them off your RAM and store in it your disk (you can save it as .npy file and later load them) so as to avoid 100% memory consumption.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.