Adding extra inputs when fine-tuning BERT
I am trying to fine-tune BERT for a sequence classification task where in addition to the sequences, I have extra features such as the writer age, tags, etc. I want to use those extra features, and I was thinking about concatenating them to the input of the final linear layer.
Is there a way of doing such a thing? If not, what is the best way for integrating extra features in the fine-tuning process?

 
You could try stacking a linear layer over-top of BERT that takes as input the BERT sequence representation + your features. You would have to fine-tune through all of BERT.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>> I am trying to fine-tune BERT for a sequence classification task where in addition to the sequences, I have extra features such as the writer age, tags, etc. I want to use those extra features, and I was thinking about concatenating them to the input of the final linear layer.
> Is there a way of doing such a thing? If not, what is the best way for integrating extra features in the fine-tuning process?

Did you find a way to add extra features then fine-tuning BERT?