Unclear how to decode a model's output
## Unclear how to decode a model's output

Hello, after digging through the docs for about an hour it's still rather unclear to me how one is supposed to decode a model's output.

Using the following code:
```
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
model = DistilBertModel.from_pretrained('distilbert-base-uncased')

input = torch.tensor(tokenizer.encode('Some text')).unsqueeze(0)
outputs = model(input)
lhs = outputs[0]
print(tokenizer.decode(lhs))
```

The lhs is always decoded as `[UNK]`

Is this just the expected result due to the model being untrained ? Is the decode functionality of the tokenizer being used in the wrong way ?

Searching for `decode` in the docs yields no code examples with it being used with a model's output.
DistilBERT as any BERT is a Transformer encoder so it encodes a sequence of tokens into a vector in the embedding space. It doesn't return a sequence of tokens.

The output of the model is `return output # last-layer hidden-state, (all hidden_states), (all attentions)` https://github.com/huggingface/transformers/blob/master/transformers/modeling_distilbert.py#L484.

If you check the size of this hidden-state, it is `torch.Size([1, 4, 768])`.
`768` being the size of the hidden-state ie the size of the embedding vector.
`4` is the number of token in input sequence (`[CLS]` and `[SEP]` tokens are added by tokenizer)
`encode` is meant to return a sequence of token from a sequence of words.
`decode` is meant to return a sequence of words from a sequence of tokens.

So if you do:

```python
encoded = tokenizer.encode('Some text')
# encoded: [101, 2070, 3793, 102]
decoded = tokenizer.decode(encoded))
# decoded: [CLS] some text [SEP]
```

But, you can't use `decode` on the output of the model as it's not a sequence of tokens but an embedding vector.

Don't hesitate to ask question if my explanation isn't clear.<|||||>Hmh, I'll try to re-phrase my question because your answer did not clear up any of my confusion:

Given the output of any hugging face model (e.g. the ones with a language modeling head, take for example `GPT2LMHeadModel`), how does one actually go from the model's output to words ?
<|||||>Hi, you can read the [quickstart](https://huggingface.co/transformers/quickstart.html#openai-gpt-2) of the documentation to see how to use `GPT2LMHeadModel` with the decoding method.<|||||>@George3d6 you can decode if the output of your model has the size of the vocabulary. So you need an output head that convert the hidden-size of the encoder or decoder into the vocabulary size. For `GPT2LMHeadModel`, follow what Lysandre said.<|||||>thank you all for your explanations here. I can do that with the GPT2 models with no issues, but my issue is that now I want to do the same but with the smaller simpler DistilmBERT model which is also multilingual in 104 languages, so I want to generate text in for example Spanish and English and with this lighter model. So I do this:

tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-multilingual-cased')
model = DistilBertForMaskedLM.from_pretrained('distilbert-base-multilingual-cased')
input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute", add_special_tokens=True)).unsqueeze(0)  # Batch size 1
outputs = model(input_ids, masked_lm_labels=input_ids)
loss, prediction_scores = outputs[:2]

but now, how do I get the continuation of the phrase at that point? I tried to apply tokenizer.decode with no luck there, thank you<|||||>This issue is quite old. But for those who still looking for the answer, you should load the model with LM Head instead.
for BERT is BertForMaskedLM.from_pretrained()
for DistilBERT is DistilBertForMaskedLM.from_pretrained()

The tokenizer is the same.