Unable to generate ALBERT embeddings of size 128
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->

Hi Team Hugging face,

Due to memory issues I wanted to migrate from BERT to ALBERT,tried the model present in the transformers,but I'm unable to generate the embedding's of size 128,all I get in outputs is 768 dimension embedding's,can you please let me know how do I get a 128 dimension embedding for any input text passed to ALBERT model.Below is the sample code,output_ids contain the embedding's at each layer.

```
from transformers import AlbertTokenizer, AlbertModel
albert_model=AlbertModel.from_pretrained('albert-base-v2',output_hidden_states=True,output_attentions=True)
albert_tokenizer=AlbertTokenizer.from_pretrained('albert-base-v2')
input_ids = tf.constant(tokenizer.encode("Hugging face is great"))[None, :]  # Batch size 1
outputs = model(input_ids)

##Displaying ALBERT config
albert_model

AlbertModel(
  (embeddings): AlbertEmbeddings(
    (word_embeddings): Embedding(30000, 128, padding_idx=0)
    (position_embeddings): Embedding(512, 128)
    (token_type_embeddings): Embedding(2, 128)
    (LayerNorm): LayerNorm((128,), eps=1e-12, elementwise_affine=True)
    (dropout): Dropout(p=0.1, inplace=False)
  )
  (encoder): AlbertTransformer(
    (embedding_hidden_mapping_in): Linear(in_features=128, out_features=768, bias=True)
    (albert_layer_groups): ModuleList(
      (0): AlbertLayerGroup(
        (albert_layers): ModuleList(
          (0): AlbertLayer(
            (full_layer_layer_norm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
            (attention): AlbertAttention(
              (query): Linear(in_features=768, out_features=768, bias=True)
              (key): Linear(in_features=768, out_features=768, bias=True)
              (value): Linear(in_features=768, out_features=768, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
              (dense): Linear(in_features=768, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
            )
            (ffn): Linear(in_features=768, out_features=3072, bias=True)
            (ffn_output): Linear(in_features=3072, out_features=768, bias=True)
          )
        )
      )
    )
  )
  (pooler): Linear(in_features=768, out_features=768, bias=True)
  (pooler_activation): Tanh()
)

```

Please format your post with [code blocks](https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks).

This seems like a very general question, where you want to change the size of a dimension of an output tensor. There are different approaches to this. If you want to do this as part of a downstream task, you can simply work with a linear layer from 768 to 128. If you are just extracting features from the model and want to reduce the dimensionality, you can do pooling, typically max or mean pooling. Since this is quite a general question, I suggest that you make a question on [Stack Overflow](https://stackoverflow.com/) instead, and close this question since it's not specific to `transformers`.<|||||>Sure,Thanks