DataCollatorForWholeWordMask does not return attention_mask
Hi,

** DataCollatorForWholeWordMask** does not output `attention_mask`. According to the `__call__` method:
`return {"input_ids": inputs, "labels": labels}`. 

Is there a peculiar motivation behind it or is a small bug? From where I see, when we do the pre-training, most instances will **not** be of the same length and applying _Attention_  for all the tokens (including the padding) may cause imprecise results.
Not sure where you are seeing this. `DataCollatorWithPadding` can also pad attention masks, as it applies `tokenizer.pad()` on the inputs as can be seen [here](https://github.com/huggingface/transformers/blob/c7faf2ccc05a095870c3c905d232179fb323797d/src/transformers/data/data_collator.py#L118).<|||||>@NielsRogge am sorry for confusing, meant **DataCollatorForWholeWordMask**, will correct that now<|||||>There is also a bug with `pad_to_multiple_of` argument: it is not passed to `_collate_batch` inside `__call__`. Have made a pull request https://github.com/huggingface/transformers/pull/12999 to add its usage.<|||||>That class should never have been merged as it is, it was a mistake on our side. It contains multiple bugs and only works for BERT models. It need to be rewritten from scratch to be model agnostic.<|||||>For me it works perfectly for any arbitrary model (checked it is doing everything correct), except for `attention_mask` generation and `pad_to_multiple_of` usage (both can be corrected manually though, however, it is always better not to invent a bicycle).<|||||>It cant' give good results on a tokenizer that is not like BERT, since it relies on the "##" to detect if something is inside a word.<|||||>Is there any progress on this issue? It seems that this issue still exists.