A dummy approach would be:
class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc = nn.Linear(4, 2)
        self.base = models.resnet50()
        
    def forward(self, x):
        # split x to x0, x1
        x0, x1 = x[:, :4], x[:, 4:]
        x0 = self.fc(x0)
        # Concatenate
        x = torch.cat((x0, x1), 1)
        # reshape to fit resnet50 input shape
        x = x.view(x.size(0), 3, 224, 224)
        x = self.base(x)
        return x

model = MyModel()
x = torch.randn(1, 3*224*224 + 2)
output = model(x)

Note that I’ve used resnet50, which expects image tensors as the input, thus I had to reshape the concatenated tensor to [batch_size, 3, 224, 224].
If you are dealing with a pretrained model using only linear layers, you wouldn’t have to reshape it.



 amirhf:

Also, is it because the output of the Keras model is a numpy array and doesn’t keep the grad_fn and requires_grad that this is not possible?


Yes, basically you would have to convert the concatenated PyTorch tensor to a numpy array, which will detach the computation graph, so that you won’t be able to calculate the gradients using the final loss.