How to get back the identified words from LayoutLMForTokenClassification?
I am using LayoutLMForTokenClassification as described [here](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/LayoutLM/Fine_tuning_LayoutLMForTokenClassification_on_FUNSD.ipynb). In the end, the tutorial shows an annotated image with identified classes for various tokens.
How can I get back the original words as well to be annotated along with the labels?

I tried to read the words with tokenizer.decode(input_ids).split(" ") but the tokenizer broke words into multiple tokens which it wasn't supposed to. So, I have more words/outputs/boxes that I am supposed to have.


Hi,

A solution for this can be the following (taken from my [Fine-tuning BERT for NER notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/BERT/Custom_Named_Entity_Recognition_with_BERT_only_first_wordpiece.ipynb)):

![image](https://user-images.githubusercontent.com/48327001/119956504-72b47100-bfa1-11eb-949a-955ede29584a.png)

In my notebook for `LayoutLMForTokenClassification`, only the label for the first word piece of each word matters. In HuggingFace Transformers, a tokenizer takes an additional parameter called `return_offsets_mapping` which can be set to `True` to return the (char_start, char_end) for each token.

You can use this to determine whether a token is the first wordpiece of a word, or not. As we are only interested in the label of the first wordpiece, you can assign its label to be the label for the entire word.

Do you understand?

<|||||>I do. Thanks. I tried doing this by referring your BERT code. But I am getting this error, unfortunately. 

![image](https://user-images.githubusercontent.com/18511921/120136599-26ab3b80-c1f0-11eb-9478-cba341955c6a.png)

Apologies if I messed up. This is the first time that I am working with transformers.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.