I assume you load all your models and push it on different GPUs?
How big is each model?
Since you have 6 models and 4 GPUs, 2 GPUs will have 2 models on them?
You are probably running out of memory on these two?
If so, you could try to relocate the models after each operation, i.e.:
def forward(self, x):
    ...
    self.ensemble_models[0].cuda(0)
    log_softmax, softmax = self.ensemble_models[0](x_normalized)
    self.ensemble_models[0].cpu()
    ...

This will obviously slow down your ensemble, so you could try running it on CPU from the beginning.
I’m still interested in the final performance, since even though each model can predict its own class, the ensemble might fail when for example two models are predicting a very high probability.
I assume you’ve trained each model separately on its positive/negative samples.