what is the most efficient way to store all hidden layers' weights?
Hi,

I am following this [post](https://mccormickml.com/2019/05/14/BERT-word-embeddings-tutorial/) for getting all 12 hidden layers' weights for every token in a sentence.

Consider I have a short text with 2 sentences: `He stole money today. He is fishing on the Mississippi riverbank.`

I want to store 5 + 8 = 13 tokens - all 12 hidden layers weights where each tensor's size=768. So, I will have 13 x 12 = 156 tensors.

I want to save all the weights in a file and I am wondering if I should use `pickle` or `hd5` format (I am working with long text documents.) I am planning to separate two sentences by a blank line, please suggest if any better ways to do it.

Thanks!
PyTorch has its [own saving utility](https://pytorch.org/tutorials/beginner/saving_loading_models.html): `torch.save`, which sounds good for your use case as you can easily save/load the tensors. It's based on pickle.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
