Body: <p>I'm currently trying to train a siamese network to determine if two inputs are similar or not. The inputs are power consumption traces and I'm basically using the siamese network as some sort of Pearson's correlation, telling me whether or not the traces are similar.</p>
  <p>After realizing that my labels were wrong, I tested using entirely random labels, and found out that the training accuracy was good (which is normal as the network can learn the dataset itself), but my problem is that the validation accuracy was also very good, which is not supposed to happen as the training and validation labels are not correlated. I have a relatively small dataset(560 traces for training and 240 for validation), but I thought the behavior of the loss and accuracy would be erratic, whereas it is relatively stable, and even repeating the training multiple times resulted in a similar loss/accuracy curve.</p>
  <p>My question is: Is it normal or did I mess up something?</p>
  <p>Here's a plot of the training/validation accuracy/loss I obtained:</p>
  <p><a href="https://i.sstatic.net/jkuA8.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jkuA8.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-05T19:57:50.253Z
Comments.Text: no, it means that you are shuffling data in a correlated way, or that you have duplicate entries between train and validation set, so that when the network overfits (and learns the answers by heart, since they are uncorrelated), it's also memorizing the validation ones
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-07T07:00:31.810Z
Comments.Text: By randomized labels, do you mean that for each *pair* you assigned a random "is similar"/"is not similar"? i.e., there's a 50%, both the validation and train set, that the output is "is similar"/"is not similar".
Comments.UserDisplayName: Alexander Wan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-08T08:08:02.170Z
Comments.Text: @AlexanderWan Yes, I assigned to the "is similar" set random traces and to the "is not similar" set traces that are not part of the first set. The weird thing is that if I assign the first 400 traces as similar and the next 400 as not similar, I do have a 50% validation accuracy, as expected. And also if I add more and more traces, the maximum validation accuracy decreases.
Comments.UserDisplayName: WINTERSDORFF Raphael
Tags: <deep-learning><siamese-neural-network>
Title: Is it possible to have a good validation accuracy with random labels?
UserDisplayName: WINTERSDORFF Raphael
ThreadText: # Is it possible to have a good validation accuracy with random labels?
  
  (Asked by: username_0 on 2023-10-05)
  
  I'm currently trying to train a siamese network to determine if two inputs are similar or not. The inputs are power consumption traces and I'm basically using the siamese network as some sort of Pearson's correlation, telling me whether or not the traces are similar.
  
  After realizing that my labels were wrong, I tested using entirely random labels, and found out that the training accuracy was good (which is normal as the network can learn the dataset itself), but my problem is that the validation accuracy was also very good, which is not supposed to happen as the training and validation labels are not correlated. I have a relatively small dataset(560 traces for training and 240 for validation), but I thought the behavior of the loss and accuracy would be erratic, whereas it is relatively stable, and even repeating the training multiple times resulted in a similar loss/accuracy curve.
  
  My question is: Is it normal or did I mess up something?
  
  Here's a plot of the training/validation accuracy/loss I obtained:
  
  [![enter image description here](https://i.sstatic.net/jkuA8.png)](https://i.sstatic.net/jkuA8.png)
  
  **Tags:** deep-learning, siamese-neural-network
  
  > **Comments:**
  > **username_1**: no, it means that you are shuffling data in a correlated way, or that you have duplicate entries between train and validation set, so that when the network overfits (and learns the answers by heart, since they are uncorrelated), it's also memorizing the validation ones  
  > **username_2**: By randomized labels, do you mean that for each *pair* you assigned a random "is similar"/"is not similar"? i.e., there's a 50%, both the validation and train set, that the output is "is similar"/"is not similar".  
  > **username_0**: @username_2 Yes, I assigned to the "is similar" set random traces and to the "is not similar" set traces that are not part of the first set. The weird thing is that if I assign the first 400 traces as similar and the next 400 as not similar, I do have a 50% validation accuracy, as expected. And also if I add more and more traces, the maximum validation accuracy decreases.  
  
  ---
  
  ## 0 answers
  
  