Not sure what you are trying to achieve here. In my case, I create two batches, one for the left images, one for the right ones, concatenate along batch dimension, then merge pairs of embedding vectors:

batch: [im_1_l, im_2_l, im_3_l] , [im_1_r, im_2_r, im_3_r]  (3 x C x H x W) x 2
concatenate: [im_1_l, im_2_l, im_3_l, im_1_r, im_2_r, im_3_r] (6 x C x H x W)
process each image with CNN: [emb_1_l, emb_2_l, emb_3_l, emb_1_r, emb_2_r, emb_3_r] (6 x E)
un-concatenate: [[emb_1_l, emb_1_r], [emb_2_l, emb_2_r], [emb_3_l, emb_3_r]] (3 x 2 x E)
merge embeddings: [y1, y2, y3] (3 x F)