transformers.TFTrainer: Does not support batches with sequences of variable lengths?
Hello,
It seems like `np.append` in `TFTrainer.prediction_loop` is the only thing that prevent TFTrainer from being able to deal with batches of variable sequence length (between the batches, not inside the batches themselves). Indeed, `np.append` requires the batches to be of the same sequence length. 
Alternatives: as this is in tensorflow, an easy alternative would be to convert the batches to `tf.RaggedTensor` with `tf.ragged.constant`, and to concatenate them (the usual way) with `tf.concat`.
You could also ofc just make `preds` and `label_ids` into lists. There doesn't seem to be any big computation going on on these objects.

Created pull request https://github.com/huggingface/transformers/pull/8485<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Hello @JulesGM 
Regarding [#8483](https://github.com/huggingface/transformers/pull/8485)
I went through the code and I found this helpful. But I'm facing issues to convert my training tf.data.Dataset to tf.RaggedTensor format. if possible can you share resources regarding this?