You should be able to create a pytorch model with each of the huggingface models initialized as layers of the model. Then in the forward function for the pytorch model, pass the inputs through self.model_a and self.model_b to get logits from both. You can concatenate these there and pass them through the rest of the model. I’ve written the PSEUDOCODE (this code won’t run directly, but presents the general idea) for the same below:
import torch.nn as nn
from transformers import AutoModel

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.model_a = AutoModel.from_pretrained('model_a')
        self.model_b = AutoModel.from_pretrained('model_b')

        self.classifier = nn.Sequential(
            nn.Dropout(p=0.1),
            nn.Linear(768, 768, bias=True),
            nn.Tanh(),
            nn.Dropout(p=0.1),
            nn.Linear(768, 3, bias=True)
        )

    def forward(self, input_ids, attention_mask):
        logits_a = self.model_a(input_ids, attention_mask=attention_mask).last_hidden_state[:, 0, :]
        logits_b = self.model_b(input_ids, attention_mask=attention_mask).last_hidden_state[:, 0, :]
        concatenated_vectors = torch.concat(logits_a, logits_b)
        output = self.classifier(concatenated_vectors)
        return output

model = Net()

You can just train this model like how you train a regular Pytorch model.
Edit: Made a small error in the code by passing x to classifier instead of concatenated_vectors.