I facing the same issue in version 4.7.0 Using   eval_accumulation_steps = 2 eventually ends up in RAM overflow and killing the process (vocabulary size is about 40K, sequence length 512, 15000 samples is about 3e11 float logits).
As a workaround I’ve added logits = [l.argmax(-1) for l in logits] immediately after prediction_step in evaluation_loop function in trainer.py  . That helps for evaluation, not sure, though, that I’ve not broken something else.