You could write a custom network (subclass of torch.nn.Module) whose forward() function computes the cosine similarity, and then use DataParallel or DistributedDataParallel to run your data through this network.
Note: This is just a hunch, I haven’t done something similar (well, except all the training I do is in fact quite similar …) so there may be pitfalls which I don’t see.