Padding part output in BERT NER task is not [PAD]?
## ❓ Questions & Help

Hello ,my friends.

I have a problem while I am trying to do a Chinese NER task.

For easy to understand I will use some English words instead.

Assume padding length =128, here is the sentence:

> [CLS] Marilyn Monroe is an famous actress. [SEP] [PAD]  [PAD]  ...  [PAD] 

After I put it into `BertForTokenClassification` , I got a output like:

>[CLS] Marilyn Monroe is an famous actress.
>[CLS] B-PER I-PER O O O O O

That looks good ,but ,when output reached padding area it become strange:

>[SEP] [PAD] [PAD] [PAD] [PAD] [PAD] ...  [PAD]
>O O O O O  [CLS] [CLS] O ...O

it seems padding area including [SEP] token output randomly (they should be [SEP] and [PAD]), and most of them are 'O' with a littel [CLS] or [B-PER]\[I-PER].

I am confused about that.

I am sure taht:

- I already set `attention_masks` on padding area while training, attention_masks on [PAD] are 0.

- I already set `token_type_ids`, token_type_ids on [PAD] are 1.

and also in eval with `attention_mask` and `token_type_ids`.

So that's what I am depressed about, I have to cut output padding area before I solve this problem, I don't think it's a good idea.

Can someone help me? :(
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
