Understand the idea of margin in contrastive loss for siamese networks I was studying siamese networks for authentication. Loss is:

Y is 0 for dissimilar pairs and 1 for similar pairs.
D_w is the distance (e.g. euclidean distance) between two pairs (by using weights w).
If pairs are similar, then loss is equal to the green box in loss function.
If pairs are dissimilar, then loss is equal to the red box in the loss function.
Suppose that we have two similar samples, then the loss is equal to the distance^2 between them. Since the aim of the training is to minimize the loss, therefore by training on similar pairs, will help to minimize the distance between similar pairs.
If we have two dissimilar samples, therefore there is an m term, which signifies margin in the loss function. According to the red box, if margin<D_w, then, loss is equal to zero. What exactly signifies the margin. What we are separating with margin? Suppose that I want to recognize face x (aim: to understand input is face x or not), what margin does? Is it something linear (like SVM margin), circle or another form? Should it have a specific form? Why if margin<D_w, loss does nothing. I do not understand this part. I do not understand if it is a learnable parameter or a pre-defined value? If it is a pre-defined value, how do we predefine it, based on which knowledge?
Edit 1: I found this explanation but I did not understand. I put it, in case it helps you to understand:
taken from here:

Role of margin m: It is to be noted that the representations of
negative pairs will only contribute to the loss if the estimated
distance ||f(x)-f(y)|| < m. Meaning that it will no longer care how
far the negative pairs x and y are once this limit reaches. So, it can
focus more on the difficult to embed points.


If you look at the Loss it is composed of two parts, 1st the loss when they are similar and second the loss when they are dissimilar.

*

*If model works very well, observation which are similar should have very small distance. Using Sqaure distance in first ensure that model is penalized if its gives high distance for similar observation (classes)


*For Dissimilar observation, if two class are seprated well enough defined by Margin then the error contirbution is zero. Lets look at the example below :
Suppose we choose margin to be 10, and we have two observation where distance between them is 19,1 lets see how loss looks like
    Error Observation 1 = max(0,10-19)^2   = 0  - No Contribution to error
    Error Observation 2 = max(0,10-1)^2    = 81  - High Contribution to error

As objective is to minimize loss, Observation 2 which is difficult to embed would be contibuting to loss so model can optimise for that
So margin ensure observation which have was well separated i.e. Distance Greater than Margin their contribution to error is zero. So optimisation algorithm can concentrate on seprating difficult Data Point in above example Observation 2. Thus Margin Helps on optimisation to embed difficult to seprate points
