Thank you for your reply!
During training I set positive samples whose labels are 1 and negative samples whose labels are 0. The positive sample includes two images which I think are similar.
The batch_size I use is 1.
I am not sure if I should put a Dropout Layer behind the FC Layer. The original purpose I do this was to avoid over-fitting. In addition, I am not sure if the output channels of FC Layer should be 2. Now I tend to set it 1.
And to my surprise, when I use the training model to do my follow-up work, the result is not bad.