Hello,
I’m trying to build a model using PyTorch + pytorch_transformers to create it using BERT as pre-training step. The issue that I’ve different datasets, which all of them are on English, but they have partial intersected labels.
It’s possible to create a model that uses pre-trained BERT (or any other model), and feeds data from multiple datasets to predict multiple outputs?
Example, which I have 4 text datasets:
Dataset A contains [ ValueA, ValueB, ValueC ]
Dataset B contains [ ValueA, ValueB, ValueC, ValueD, ValueE, ValueF ]
Dataset C contains [ ValueA, ValueB ]
Dataset D contains [ ValueD, ValueE, ValueF ]
Since all of them are on English, I hope to use BERT to enchance the similarity between datasets.
Approaches that I thought:

Create a general y, and add 0. to empty fields which I don’t have for it. In this case, my prediction would be [ ValueA, ValueB, ValueC, ValueD, ValueE, ValueF ]