Hi @saireddy,
BERT supports sentence pair classification out-of-the-box. There’s no need to ensemble two BERT models.
In BERT, 2 sentences are provided as follows to the model:
[CLS] sentence1 [SEP] sentence2 [SEP] [PAD] [PAD] [PAD] …
You can prepare them using BertTokenizer, simply by providing two sentences:
from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")

sentence_a = "this is a sentence"
sentence_b = "this is another sentence"

encoding = tokenizer(sentence_a, sentence_b, padding="max_length", truncation=True)

This encoding is a Python dictionary, which includes keys including input_ids. If you decode them back as follows:
tokenizer.decode(encoding["input_ids"])
Then this will print [CLS] this is a sentence [SEP] this is another sentence [PAD] [PAD] [PAD] …