HI Armel,
if you the problem you are trying to solve is: given sentence A and Sentence B, are they a paraphrase of each other?
is this it?
if so, you can use this: Solve GLUE tasks using BERT on TPU  |  Text  |  TensorFlow 7
more specifically the QQP benchmark.