Thanks for your detailed comments.
I believe I can here provide you with a command to reproduce this issue. (Unless I am somehow confused.)
According to my reading of your comments, this command, using the xsum dataset, should (but does not) generate summaries of fewer than 50 tokens:
RUN="xsum-1500-train-try-max-len"

python3 /workspace/rabbit-py/transformers/examples/seq2seq/finetune_trainer.py \
    --learning_rate=3e-5 \
    --fp16 \
    --do_train --do_eval --do_predict \
    --evaluation_strategy steps \
    --predict_with_generate \
    --n_train 1500 \
    --n_val 300 \
    --n_test 100 \
    --num_train_epochs 1 \
    --data_dir "/workspace/rabbit-py/corpii_foreign/xsum" \
    --model_name_or_path "t5-small" \
    --output_dir "/workspace/rabbit-py/predictions/$RUN" \
    --per_device_train_batch_size 5 \
    --per_device_eval_batch_size 8\
    --task 'summarization' \
    --overwrite_output_dir \
    --run_name $RUN \
    --val_max_target_length 50 \
    --test_max_target_length 50 \
    --max_target_length 50 \
    "$@"

When I run this, I get summaries like
the trio are up for best UK act and best album, as well as two nominations in the best song category. they have been nominated for their favourite album, Number One and Strong Again.
Which I think are too long.
I also see these lines in the console output:
 INFO - utils -   using task specific params for summarization: {'early_stopping': True, 'length_penalty': 2.0, 'max_length': 200, 'min_length': 30, 'no_repeat_ngram_size': 3, 'num_beams': 4, 'prefix': 'summarize: '}
"task_specific_params": {
    "summarization": {
      "early_stopping": true,
      "length_penalty": 2.0,
      "max_length": 200,
      "min_length": 30,
      "no_repeat_ngram_size": 3,
      "num_beams": 4,
      "prefix": "summarize: "
    },

thanks again