I have model A, which is BERT, and model B, which is also BERT. Each model has different input features.
I removed the classifier layer from each model before constructing a parent model in which I will perform concatenation for three distinct features:
-The output of model A (text)
-The output of model B (POS)
-And other features (subjectivity)
Then I will make it input to the fully connected layer and make the classification
The architecture will be like this

Screen Shot 2021-11-02 at 2.50.31 PM1592×1080 56.5 KB

the problem is that the loss too high 218 and the accuracy not improved compared to the results before adding the new features which is not input to any model…
I think I should do normalization but I don’t know how…
This is the parent model:
class ParentModel(torch.nn.Module):
def __init__(self, modelA, modelB):
    super(ParentModel, self).__init__()
    self.modelA = modelA
    self.modelB = modelB
    self.fc1 = torch.nn.Linear(2560, 1536)
    self.classifier = torch.nn.Linear(1536, 3)
    
def forward(self, x1, x2, x3, x4, x5, x6):
    x11 = self.modelA(x1, x2)
    x22 = self.modelB(x3, x4)
    x = torch.cat((x11, x22, x5, x6), dim=1)
    x = torch.nn.ReLU()(self.fc1(x))

    x = self.classifier(x)

    return x

ids1 = batch1[‘ids’].to(device, dtype = torch.long)
mask1 = batch1[‘mask’].to(device, dtype = torch.long)
targets1 = batch1[‘targets’].to(device, dtype = torch.long)
ids2 = batch2[‘ids’].to(device, dtype = torch.long)
mask2 = batch2[‘mask’].to(device, dtype = torch.long)
ids3 = batch3[‘ids’].to(device, dtype = torch.long)
mask3 = batch3[‘mask’].to(device, dtype = torch.long)
outputs = model(ids1, mask1, ids2, mask2, ids3, mask3)
Please advise if the problem is not related to the normalization issue.
this is the results:

Screen Shot 2021-11-02 at 6.53.48 PM942×682 147 KB

The results for concatenating just the output of model A and model B is:

Screen Shot 2021-11-02 at 7.57.07 PM950×498 45.1 KB