If you are concatenating the activations at dim0, you are increasing the batch size, thus not using an ensemble.
Ensemble models would use the same input and yield an activation output or prediction, which is then either further processed in another stage or processed via a mean/median calculation. There are of course different ways to average the predictions of different models, but your current approach would not use the ensemble idea.
What are the shapes of x1 and x2 and what do the dimensions represent?