You could train the 5 classifiers separately as binary classifiers and try to combine them afterwards as shown in your figure.
For this you could pass all pretrained models to the linked ensemble model (maybe remove the last classification layer of each model), and add a classifier head on top of their concatenated outputs.
Would that work or are you thinking about another approach?