is it possible to split the embedding layers from bert during fine tuning and inference
in my bert model, parameters in embedding layers are huge since the vocabulary size is very big. So I plan to split the embedding layer from the entire model and use it as a independent module to calculate word embeddings. Is it possible to do like this? thanks

Hello, 

You can get the embeddings layer with model.embeddings as per 
https://github.com/huggingface/transformers/blob/v4.18.0/src/transformers/models/bert/modeling_bert.py#L848

```
print (model.embeddings)
#BertEmbeddings(
  (word_embeddings): Embedding(30522, 768, padding_idx=0)
  (position_embeddings): Embedding(512, 768)
  (token_type_embeddings): Embedding(2, 768)
  (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
  (dropout): Dropout(p=0.1, inplace=False)
)
```

Bonus: Could also just extract the embedding output as follows without splitting, I'm not sure why you are splitting?
https://github.com/huggingface/transformers/issues/1827#issuecomment-554028292

<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.