Probably a bit late but hope this still helps someone 
In the forward function instead of first passing the query, pos and neg into the model and then cloning the tensors I would try and pass clones of the tensors in:
out_q = self.net(query.clone())