run_clm.py script.
Usually, the next token is predicted by sampling from the logits of the last hidden state the model produces from the
input sequence.
Here is an example of using the tokenizer and model and leveraging the
[PreTrainedModel.top_k_top_p_filtering] method to sample the next token following an input sequence
of tokens.
```py
from transformers import AutoModelForCausalLM, AutoTokenizer, top_k_top_p_filtering
import torch
from torch import nn
tokenizer = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2")
sequence = f"Hugging Face is based in DUMBO, New York City, and"
inputs = tokenizer(sequence, return_tensors="pt")
input_ids = inputs["input_ids"]
get logits of last hidden state
next_token_logits = model(**inputs).logits[:, -1, :]
filter
