In general, context refers to the input that a model has to make a prediction on (e.g., the beginning of a sentence in a next word prediction task). So, I should ask, what sort of task are you training on? Is it just span corruption prediction (i.e., the original T5 training objective)?
Regarding your code. One does not expect the generate output to be identical to the training output. One reason for that is that the generate API is used to perform a task in inference mode (i.e., after we finished training the model). This means that layers that behave different in training vs inference (e.g., dropout) will have different behaviours so the model will not produce the same output.
So, I suggest fine tuning the model on your task and then calling the generate API. It could be that you are checking the behaviour early on when the model has not had time to learn anything.