Zero shot multilingual BERT
❓ Questions & Help

I have a doubt about the usage of multilingual BERT.

I did a domain adaptation on the language model **[`BERT-Base, Multilingual Cased`](https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip)** with a dataset of a slightly different kind of text. This dataset is unbalanced on English language but contains also other languages as Italian.

Then I did a finetuning on an Italian dataset on NER.

Has this kind of training a name, like **zero shot** classification (beacuse I adapted on a multilingual dataset but unbalanced on English and then we finetune on Italian)?
Or maybe the 0 shot would be in the case in which I finetuned with an Italian dataset and then I evaluated on a corpus of another language?

Thanks
I think what you did there is probably best described as transfer learning.

> Or maybe the 0 shot would be in the case in which I finetuned with an Italian dataset and then I evaluated on a corpus of another language?

Yeah, this is closer to the way the term "zero-shot" is being used in the field right now. Here's a [recent example](https://arxiv.org/abs/1812.10464). 

I'll also note that the way I've seen "zero shot learning" used traditionally was pretty narrow: it meant training a classifier on one set of labels and then evaluating on a different set of labels on in-domain data. Recently, especially in NLP, it's often been used more broadly to mean "do a task that the model wasn't explicitly trained on without additional fine tuning", e.g. in the GPT-2 paper.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
