Generating text with fine-tuned TFGPT2LMHeadModel in python. 
I've finetuned GPT2 using the following script:
```
from transformers import GPT2Tokenizer, TFGPT2LMHeadModel
import tensorflow as tf

model = TFGPT2LMHeadModel.from_pretrained("distilgpt2")
tokenizer = GPT2Tokenizer.from_pretrained("distilgpt2")

file_path = 'text.txt'
with open(file_path, encoding="utf-8") as f:
    text = f.read()

tokenized_text = tokenizer.encode(text)

examples = []
block_size = 100
for i in range(0, len(tokenized_text) - block_size + 1, block_size):  # Truncate in block of block_size
    examples.append(tokenized_text[i:i + block_size])

inputs, labels = [], []
for ex in examples:
    inputs.append(ex[:-1])
    labels.append(ex[1:])

dataset = tf.data.Dataset.from_tensor_slices((inputs, labels))

BATCH_SIZE = 8
BUFFER_SIZE = 10000

dataset = dataset.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)

optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
model.compile(optimizer=optimizer, loss=[loss, *[None] * model.config.n_layer], metrics=[metric])
model.fit(dataset, epochs=20)
```
This runs fine, and after 20 epochs I have an accuracy of ~0.59. 
The problem comes when I tried to write my own text generation script:
```
def generate_text(model, tokenizer, start_string, num_generate):

        input_eval = tf.expand_dims(tokenizer.encode(start_string), 0)
        token_ids = []

        for i in range(num_generate):
            predictions = tf.squeeze(model.predict(input_eval)[0], 0)

            predicted_id = tf.random.categorical(predictions, 1)[-1, 0].numpy().item()
            
            input_eval = tf.expand_dims([predicted_id], 0)

            token_ids.append(predicted_id)
    
        return start_string + tokenizer.decode(token_ids)
```
I get output, but the output is of a sufficiently lower quality than when I train a model using "run_lm_finetuning.py" and generate text using "run_generation.py"
I looked into the example generation script, and it looks like there is simply a call to "model.generate(...)"

Where does this model.generate() method exist? 
Did you resolve this? I think on the current commit `generate()` still doesn't exist.<|||||>There is no TensorFlow implementation for the `generate()` method yet. We're working on it, but in the meantime, you could do your own generation loop or use a PyTorch model with the `generate()` method.<|||||>I need to change to `BATCH_SIZE = 12` in the above or else this example code will not run.  There would be a dimension mismatch with `BATCH_SIZE = 8`