ptrblck:

You could just create several different models, push them onto the GPU and feed your data.


So, as I understood:

I save a model this way

# ... some code here
torch.save(model.state_dict(), "{}.pt".format(output_name))
# NOTE: output_name is modelX_name in next steps


Then I load model this way

model1 = ... # your model1. E.g, model1 = CatDogClassifier().to('cuda')
model2 = ... # your model2
model1.load_state_dict(torch.load(model1_name))
model2.load_state_dict(torch.load(model2_name))


And If I want to make a prediction I simply type

X = ... #some data. 
# e.g, X = torch.tensor(X, requires_grad=False, dtype=torch.float).to('cuda')
out1 = torch.max(model1(X), 1)[1]
# and for numpy output if you use 'cuda':
# out1 = torch.max(model1(X), 1)[1].cpu().numpy()

Please, correct me if I’m wrong. Can you provide something like pseudocode if I’m missed something?