I found that during evaluation the <s> tokens are still decoded by my tokenizer :
"<s>c<s>e<s>c<s>i<s> est <s>un<s> <s>t<s>e<s>s<s>t<s>e<s>"instead of "ceci est un teste "
I suppose the tokens are ignored during the compute of the CTC loss but not for the eval metrics.
To correct that I changed in compute_metrics(pred) pred_str = tokenizer.batch_decode(pred_ids) by pred_str = tokenizer.batch_decode(pred_ids, skip_special_tokens=True).
Is there a way to add this parameter to the tokenizer config and not during the decoding call? I didn’t find anything about that.
It would also allow to have the output nicely decoded when model is tested in the hub.