The huggingface documentation of T5 (see Training) says:

The PAD token is hereby used as the start-sequence token.

Since decoder_start_token_id is an optional parameter for T5, I also tried it without setting it at all - but with the same result (I suspect the pad token is set as default).