Great, it looks much simpler than I expected 
Why can’t I do it in a batched way? That’s because I hid a lot of complexity in this example, in reality I extract a random number of descriptors from each batch element so the pseudo-distance matrices are of different shapes from each other (which would require a huge amount of padding) and even if that worked, the loss_fn uses external, non-batchable operations.
Thanks a lot!