run_generation.py - use of < redirect for input text file only reads first line - solution requested
Example:

python3 examples/run_generation.py --model_type=gpt2 --model_name_or_path="models/custom-774M" --length=400 < test.txt > test_gpt2_400.txt

Version 2.0.3 and earlier accepts a simple linux redirect < test.txt and reads the file line-by-line while generating the corresponding output to > test_gpt2_400_out.txt

This works perfectly and does not require any changes. The test.txt is a simple 3 line text file with 1 sentence on each line.

Versions 2.0.4 to 2.0.8 do not accept the < redirect in the same manner. The above only reads the first line of 'test.txt', generates, then exits. Output is the same for the first line.

Is there a simple change to the 2.0.8 version of 'run_generation.py' that will return this useful terminal function under ubuntu 18.04?

Have tried 'while - read' loops and only managed to generate the Model prompt >>> 3x requesting terminal input:

exec 3<"test.txt"

while IFS= read -r line <&3; do

python3 examples/run_generation.py --model_type=gpt2 --model_name_or_path="models/custom-774M" --length=400 

done > test_gpt2_400_out.txt

Any help is appreciated.
I think in your case it might make more sense to use the `TextGenerationPipeline`. It's a two liner that does the same as the script. You can wrap a `open(inpt_text_file)` and `file.write()` around these two lines in your use case. 

See documentation for the generation pipeline here: https://huggingface.co/transformers/usage.html#text-generation<|||||>Closing for now