Hi! Can you be a little bit more specific about your query?
Just to give you a head start,
In general, NER is a sequence labeling (a.k.a token classification) problem.
The additional stuff you may have to consider for NER is, for a word that is divided into multiple tokens by bpe or sentencepiece like model, you use the first token as your reference token that you want to predict. Since all the tokens are connected via self-attention you won’t have problem not predicting the rest of the bpe tokens of a word. In PyTorch, you can ignore computing loss 10 (see ignore_index argument) of those tokens by providing -100 as a label to those tokens (life is so easy with pytorch  ).
Apart from that, I didn’t find any more additional complexity in the training NER model.
Some other implementation details you need to check,

One important Note: So far I remember (please verify), In conll, german or dutch dataset there are 2-3 long sentences in the test dataset. Sequence labeling doesn’t work like sentiment analysis. You need to make sure your sentence is not cut down by the max_sequence_len argument of the Language Model’s tokenizer. Otherwise, you will see a little bit of discrepency in your test F1 sore. An easier hack for this problem is to divide the sentence into smaller parts and predict them one by one and finally merge them.
Imo Self-attention and CRF layer is theoretically different but in application some of the problem that CRF solved in prior model, self-attention can also solve them (because they create a fully connected graph). So using softmax is more preferable than a CRF layer.
The score that the original BERT paper 2 reported are not reproducible and comparable with most of the papers since they used document level NER fine-tuning.

If you still have query about the architecture you can follow this,
  
      Guillaume Genthial blog – 5 Apr 17
  
  
    

Sequence Tagging with Tensorflow 67

GloVe + character embeddings + bi-LSTM + CRF for Sequence Tagging (Named Entity Recognition, NER, POS) - NLP example of bidirectionnal RNN and CRF in Tensorflow


  
  
    
    
  
  


you only have to replace hierarchical rnn with transformer as the encoder.
You can check the following paper’s for more info,

Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT - ACL Anthology 2
https://arxiv.org/pdf/2007.07683.pdf 3
https://arxiv.org/pdf/2004.12440.pdf 3

https://arxiv.org/pdf/2004.13240.pdf 9 (beware, this is my publication. I may be biased, still not accepted)

Please let me know if you have more queries.