I thought the x1, x2, x3 ,x4 and x5 s are the outputs from the BERTClassA to BERTClassE. As defined in BERTClassA:
class BERTClassA(torch.nn.Module):
  def init(self):
    super(BERTClassA, self).init()
    self.l1 = BertModel.from_pretrained('bert-base-uncased')
    self.pre_classifier = torch.nn.Linear(768, 768)
    self.dropout = torch.nn.Dropout(.3)
  
  def forward(self, input_ids, attention_mask):
    output_1 = self.l1(input_ids=input_ids, attention_mask=attention_mask)
    hidden_state = output_1[0]
    pooler = hidden_state[:, 0]
    pooler = self.pre_classifier(pooler)
    pooler = torch.nn.ReLU()(pooler)
    output = self.dropout(pooler)
    return output

I am asking to inherit the output from BERTClassA and give me output as x1, same goes for the next 4 BERT classes. Am i correct?
The model should conisder the input_ids and attention_masks as i defined them in the BERTClassA- BERTClassE as below:
def forward(self, input_ids, attention_mask):
    output_1 = self.l1(input_ids=input_ids, attention_mask=attention_mask)

Somehow i need to connect this to the my ensemble model.
I am a bit confused at this stage…