Could you check. If the outputs converge towards a constant value, e.g. a zero tensor?
In that case all distances would be zero and only the margin would push your loss to 0.5.
Could you try to lower the learning rate a bit and check, if the loss decreases further?