Input_embeds for Albert
Hello! I am checking the source codes for Albert's inputs_embeds. But I think the open sourced codes do not match with the document:

In the document, the description for inputs_embeds says that the shape should be the of shape (batch_size, sequence_length, hidden_size).

But in the source codes(https://huggingface.co/transformers/v3.4.0/_modules/transformers/modeling_albert.html) we have the followings:

self.position_embeddings = nn.Embedding(config.max_position_embeddings, config.embedding_size)
self.token_type_embeddings = nn.Embedding(config.type_vocab_size, config.embedding_size)
position_embeddings = self.position_embeddings(position_ids)
token_type_embeddings = self.token_type_embeddings(token_type_ids)
embeddings = inputs_embeds + position_embeddings + token_type_embeddings


So in order to make the final additions' operations valid, the shape between inputs_embeds and the other two should be the same. But the default embedding size is 128 while the default hidden size is 768("albert-base-v2"). So if we use the method described in document, the compiler will output errors.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.