bert（+lstm）+crf
add crf layer for better performance in NER tasks
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2249?src=pr&el=h1) Report
> Merging [#2249](https://codecov.io/gh/huggingface/transformers/pull/2249?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/ac1b449cc938bb34bc9021feff599cfd3b2376ae?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2249/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2249?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #2249   +/-   ##
=======================================
  Coverage   79.82%   79.82%           
=======================================
  Files         131      131           
  Lines       19496    19496           
=======================================
  Hits        15562    15562           
  Misses       3934     3934
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2249?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2249?src=pr&el=footer). Last update [ac1b449...ea25498](https://codecov.io/gh/huggingface/transformers/pull/2249?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
<|||||>@michael-wzhu did this increase f1 on CoNLL?<|||||>> add crf layer for better performance in NER tasks

Hi. How do you solve the tokens with "##..." when they are fed into the crf layer?
e.g. De ##duct ##ive reasoning
Do "##duct" and ""##ive" are fed into the crf layer? If they are, do they have chance to be transfered in the transition matrix?<|||||>> e" are fed into the crf layer? If they are, do they have chance to be transfered in the transition matrix?

I have used this to develop my own version. To answer your quesiton, ## sub word tokens are treated as padding, as suggested by the original BERT authors. This code relies on The padding label token being "X", at the first position (to get0th index) from the output of get_labels function in crf_utils_ner.py 

The pad token label id might need to be 0 for calculations in CRF, but you should be careful in declaring your mask so your model does not confuse padding with one of the tokens. <|||||>If ## sub word tokens are treated as padding, it will break the tag-tag dependencies, so definitely not ideal.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>@srush @mezig351  What did you find after your work on this issue? Why didn't you merge it?<|||||>I think you should add it, it's not trivial, and I myself spent 2 days to make it work...
and then I just found this thread... 