Solved by ptrblck in post #4 
              
              
                A dummy approach would be: 
class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.fc = nn.Linear(4, 2)
        self.base = models.resnet50()
        
    def forward(self, x):
        # split x to x0, x1
        x0, x1 = x[:, :4], x[:, 4:]
        x0 …