Answers.Body: <p>The short answer is no, you shouldn't do that.</p>
  <p>There is a &quot;distribution shift&quot; thing when you have different x-y relation on the validation set then on the train set. The distribution shift would deteriorate your model performance and you should try to avoid that. The reason it's bad - ok, you find the way to fix the model for validation data, but what about novel <em>test</em> data? Will it be like a train? Will it be like validation? You don't know and your model is worthless in fact.</p>
  <p>What you can do</p>
  <ol>
  <li>You should redefine the train/validation scheme. I would recommend making <a href="https://en.wikipedia.org/wiki/Cross-validation_(statistics)" rel="nofollow noreferrer">cross-validation</a> with at least three splits</li>
  <li>As mentioned in a comment, your model seems to overfit; try to apply some regularization techniques, like dropout, batch norm, data augmentation, model simplification and so on</li>
  <li>Add some data if it's possible, that would better cover all the possible cases.</li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-24T13:04:16.653Z
Answers.Text: Here is the source code https://github.com/Julius-R-UI/Heart-Failure-Predictions-Model
Answers.UserDisplayName: jr123456jr987654321
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-24T15:25:14.083Z
Answers.Text: 1. Have you tried logistic regression on this data as a baseline? 2. Are you sure your class balanced? If not, use roc-auc score, not accuracy.
Answers.UserDisplayName: Kirill Fedyanin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-24T15:26:03.117Z
Answers.Text: And if I answered your question, consider accepting the answer
Answers.UserDisplayName: Kirill Fedyanin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-24T13:00:53.597Z
Answers.LastActivityDate: 2021-03-29T22:23:47.633Z
Answers.LastEditDate: 2021-03-29T22:23:47.633Z
Answers.UserDisplayName: Kirill Fedyanin
Answers.Body: <p>I'm new to all this so take what I say with a grain of salt and not as fact, I don't have any formal education or training.  I believe when you're referring to inversion predictions, you're not overthinking you're underthinking.  For anything to have value it must also have an inverse or else there's no way to cognitively perceive it (contrast) otherwise you're looking at a white paper against white paper.  Now since you're referring to data set prediction, you need to define it linearly (x, y) or f(x) to scale and plot.  Therefore x and y BOTH must retain inverse proportionate values (I made up that term) in order to, in the context of assigning value, exist.  So you need to have 4 quadrants of data for predictive, so now you're looking at quantum data processing in order to facilitate predictions in a non-linear context.  Use a matrix, I believe Diroches Matrix should be applicable here.  Also, remember that predictions are always changing and updating based on empirical and real-time data, so don't get your programming stuck in an ONLY RIGHT NOW mindset, matrices are designed to be constantly moving and evolving.  Therefore your z-axis should always retain a state of variability, or it should always be Z don't attach a value to it.  Good luck.  I'm jealous I would love to ACTUALLY be working on something cool :/</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-24T22:26:17.013Z
Answers.LastActivityDate: 2021-03-31T22:20:51.723Z
Answers.LastEditDate: 2021-03-31T22:20:51.723Z
Answers.UserDisplayName: Bobby Letendre
Body: <p>I have a binary classification problem.</p>
  <p>My neural network is getting between 10% and 45% accuracy on the validation set and 80% on the training set. Now, if I have a 10% accuracy and I just take the opposite of the predicted class, I will get 90% accuracy.</p>
  <p>I am going to add a KNN module that shuts down that process if the inputted data is, or is very similar to the data present in the data set.</p>
  <p>Would this be a valid approach for my project (which is going to go on my resume)?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-24T11:40:39.517Z
Comments.Text: At a guess without more information, this sounds like a relatively severe case of overfitting, or perhaps the test data is distributed rather differently to the training data. Although in principle what you're saying would work, it sounds like a bit of a hack rather than a genuine solution to the problem. Perhaps focus on trying to figure out why you're getting such a dramatic discrepancy? We might be able to help here too.
Comments.UserDisplayName: htl
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-24T12:56:44.003Z
Comments.Text: @htl ok sure I have been working on it for weeks and nothing works. If u wanna find the source code check out this link https://github.com/Julius-R-UI/Heart-Failure-Predictions-Model
Comments.UserDisplayName: jr123456jr987654321
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-24T23:12:30.603Z
Comments.Text: Investors do this.  They call it 'contrarian trading'.  George Costanza, on the Seinfeld show, did this quite successfully for many decisions in his life: https://www.youtube.com/watch?v=cKUvKE3bQlY
Comments.UserDisplayName: Brian O'Donnell
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-26T17:28:50.740Z
Comments.Text: The part "KNN module" is not clear to me. What do you mean by that sentence? Do you mean that you will use the k-nearest neighbour to compute what exactly?
Comments.UserDisplayName: nbro
Tags: <neural-networks><deep-learning><overfitting><accuracy><binary-classification>
Title: Could I just choose the other (non-predicted) class when the accuracy is low?
UserDisplayName: jr123456jr987654321
ThreadText: # Could I just choose the other (non-predicted) class when the accuracy is low?
  
  (Asked by: username_0 on 2021-03-24)
  
  I have a binary classification problem.
  
  My neural network is getting between 10% and 45% accuracy on the validation set and 80% on the training set. Now, if I have a 10% accuracy and I just take the opposite of the predicted class, I will get 90% accuracy.
  
  I am going to add a KNN module that shuts down that process if the inputted data is, or is very similar to the data present in the data set.
  
  Would this be a valid approach for my project (which is going to go on my resume)?
  
  **Tags:** neural-networks, deep-learning, overfitting, accuracy, binary-classification
  
  > **Comments:**
  > **username_1**: At a guess without more information, this sounds like a relatively severe case of overfitting, or perhaps the test data is distributed rather differently to the training data. Although in principle what you're saying would work, it sounds like a bit of a hack rather than a genuine solution to the problem. Perhaps focus on trying to figure out why you're getting such a dramatic discrepancy? We might be able to help here too.  
  > **username_0**: @username_1 ok sure I have been working on it for weeks and nothing works. If u wanna find the source code check out this link https://github.com/Julius-R-UI/Heart-Failure-Predictions-Model  
  > **username_2**: Investors do this.  They call it 'contrarian trading'.  George Costanza, on the Seinfeld show, did this quite successfully for many decisions in his life: https://www.youtube.com/watch?v=cKUvKE3bQlY  
  > **username_3**: The part "KNN module" is not clear to me. What do you mean by that sentence? Do you mean that you will use the k-nearest neighbour to compute what exactly?  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2021-03-24. Score: 2 | Accepted answer
  
  The short answer is no, you shouldn't do that.
  
  There is a "distribution shift" thing when you have different x-y relation on the validation set then on the train set. The distribution shift would deteriorate your model performance and you should try to avoid that. The reason it's bad - ok, you find the way to fix the model for validation data, but what about novel *test* data? Will it be like a train? Will it be like validation? You don't know and your model is worthless in fact.
  
  What you can do
  
  1. You should redefine the train/validation scheme. I would recommend making [cross-validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics)) with at least three splits
  2. As mentioned in a comment, your model seems to overfit; try to apply some regularization techniques, like dropout, batch norm, data augmentation, model simplification and so on
  3. Add some data if it's possible, that would better cover all the possible cases.
  
  > **Comments:**
  > **username_0**: Here is the source code https://github.com/Julius-R-UI/Heart-Failure-Predictions-Model  
  > **username_4**: 1. Have you tried logistic regression on this data as a baseline? 2. Are you sure your class balanced? If not, use roc-auc score, not accuracy.  
  > **username_4**: And if I answered your question, consider accepting the answer  
  
  ---
  
  ### Answer by username_5 on 2021-03-24. Score: 0
  
  I'm new to all this so take what I say with a grain of salt and not as fact, I don't have any formal education or training. I believe when you're referring to inversion predictions, you're not overthinking you're underthinking. For anything to have value it must also have an inverse or else there's no way to cognitively perceive it (contrast) otherwise you're looking at a white paper against white paper. Now since you're referring to data set prediction, you need to define it linearly (x, y) or f(x) to scale and plot. Therefore x and y BOTH must retain inverse proportionate values (I made up that term) in order to, in the context of assigning value, exist. So you need to have 4 quadrants of data for predictive, so now you're looking at quantum data processing in order to facilitate predictions in a non-linear context. Use a matrix, I believe Diroches Matrix should be applicable here. Also, remember that predictions are always changing and updating based on empirical and real-time data, so don't get your programming stuck in an ONLY RIGHT NOW mindset, matrices are designed to be constantly moving and evolving. Therefore your z-axis should always retain a state of variability, or it should always be Z don't attach a value to it. Good luck. I'm jealous I would love to ACTUALLY be working on something cool :/
  
  
  ---
  
  