I tried to train without a prefix at all at first and t5 didn’t seem to handle that too well. Not that it didn’t work at all, it just didn’t work nearly as well for me than when I added the prefix. It seems to expect to parse out a prefix and base the rest of what it does fairly heavily on that.
Would suggest doing:
input ids = (Your custom prefix here): (input)
labels = (output)
You can play with different prefixes. If it describes the task (paraphrase), you may get better results earlier in training but your data is large enough that I think you can get away with using anything and as long as it’s consistent within the training run and you train long enough, it should work and produce similar results.