Can you give some suggestion about add features with input_ids to token-classification model ?
Hi, I want to add POS TAG labels as input with input_ids as auxiliary feature feature in NER model.
How can i get the entry in the forward function ?
And i review some implementations about model use bert model to perform NER  such as 
https://github.com/monologg/JointBERT/blob/master/model/modeling_jointbert.py
It seems if i want to ad this feature (POS TAG) i must rewrite the forward function in BertModel
and seems i should train the edit model from scratch.
Can you give some suggestions about edit model structure and partial weights load from 
huggingface pre-trained-models ? 

The more general question is that dose huggingface model support some interface about
add other char level feature as  auxiliary features ?<|||||>And i search some bert with ner implementations, they all not use pos tag features.
But when i view the following code :
**https://github.com/sberbank-ai/ner-bert/blob/master/modules/models/bert_models.py**
i think if add pos tag feature in lstm level (use bert embedding as input)
it seems more suitable.
Do you think this is a general solution to combine token level feature with huggingface features.
Or the future releases will support some feature combination model above bert constructions 
as options ? <|||||>Or i can use set_input_embeddings and
https://github.com/plasticityai/magnitude
Additional Featurization 
to the original embedding?<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.