Using BERT with custom QA dataset
Hi,

I want to use BERT to train a QA model on a custom SQuAD-like dataset. Ideally, I would like to leverage the learning from the SQuAD dataset, and add fine-tuning on my custom dataset, which has specific vocabulary.

What is the best way to do this?
I think that you should start by pretraining a BERT model on SQuAD to give it a sense on how to perform question answering and then try finetuning it to your task. This may already give you good results, if it doesn't you might have to dig a bit deeper in the model.

I don't really know how adding your domain specific tokens to the vocabulary would interact with the tokenizer.<|||||>One nice recent example is "A BERT Baseline for the Natural Questions" by Chris Alberti, Kenton Lee and Michael Collins from Google Research: http://arxiv.org/abs/1901.08634<|||||>This might help other dev who want to use BERT for custom QA: https://github.com/cdqa-suite/cdQA