Answers.Body: <p>In machine learning, we can use all the datasets as training data in a model.  But if there are too many data sets, or too much data, and we do not split them up, our model may be not produce acceptable results. </p>
  
  <p>Why? </p>
  
  <p>Because if the model studies too much training data, it may be <a href="https://en.wikipedia.org/wiki/Overfitting" rel="nofollow noreferrer">overfitted</a>.  </p>
  
  <p>(Just like when you cram for a test, and get overloaded with too much information!) </p>
  
  <p>What I mean is, your model is only familiar with the data you provide, not for the new data. </p>
  
  <p>So we need to use test data to train our algorithm.  Naive Bayes and Decision Tree Classifier are no exception because they can produce an overfitted model based on train data. </p>
  
  <p>So we test it on the data test to know how well the method works in relation to the problem.</p>
  
  <blockquote>
    <p>Most data scientists divide their data (with answers, that is historical data) into three portions: training data, cross-validation data and testing data. The training data is used to make sure the machine recognizes patterns in the data, the cross-validation data is used to ensure better accuracy and efficiency of the algorithm used to train the machine, and the test data is used to see how well the machine can predict new answers based on its training.</p>
  </blockquote>
  
  <p><br>
  <sub>SOURCE: <a href="https://www.researchgate.net/post/What_is_training_and_testing_data_in_machine_learning" rel="nofollow noreferrer">https://www.researchgate.net/post/What_is_training_and_testing_data_in_machine_learning</a></sub></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-21T11:15:30.850Z
Answers.LastActivityDate: 2020-04-23T21:56:33.360Z
Answers.LastEditDate: 2020-04-23T21:56:33.360Z
Answers.UserDisplayName: Muhammad Mukhtar
Answers.Body: <p>Your assumption about the test data is not correct completely. Maybe you use the test data to tune your learning algorithm to work better on the test data, but it's not the whole thing. Sometimes you need to know that the ML method is working or not and have a sense about how much does it work!</p>
  
  <p>You have other scenarios that you want to evaluate your method:</p>
  
  <ol>
  <li><p>Compare the result of the leaner with other techniques. For example, you are considering DT versus an SVM classifier over a data set. If you want to compare them, you need a value to found such a sense about the comparison.</p></li>
  <li><p>Sometimes you are using an ensemble method and you want to tune some parameters to balance between using different ML methods. Hence, you need to evaluate these learning methods (DT, Naive Bayes) to improve the ensemble method.</p></li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-07T17:09:13.467Z
Answers.LastActivityDate: 2020-04-07T17:09:13.467Z
Answers.UserDisplayName: OmG
Answers.Body: <p>One way to test the accuracy of trained model is by testing it using data test. By testing we are able to check the accuracy. Whether your model is a good model or not depends heavily on the accuracy, if your accuracy is too low or too high (eg. up to 99%~100%), there could be some problem in your model.</p>
  
  <p>For further information on the example in data test, you can access <a href="https://jakevdp.github.io/PythonDataScienceHandbook/05.05-naive-bayes.html" rel="nofollow noreferrer">https://jakevdp.github.io/PythonDataScienceHandbook/05.05-naive-bayes.html</a></p>
  
  <p>Hope this helps</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-21T09:59:00.427Z
Answers.LastActivityDate: 2020-04-21T09:59:00.427Z
Answers.UserDisplayName: Jaish Muhammad
Answers.Body: <p>I actually pondered this question a few months ago, so i understand your point of view!</p>
  
  <p>You are correct in assuming that if you already build your tree or calculate your probabilities, what is the point of using test data? Because your model is fixed the way they are, no matter if you use test data or not. Well, the purpose of test data is not only to test your model against unseen data and get some evaluation score. But it is also to test if your model is the right fit for your problem.</p>
  
  <p>One of the main reason why we all build ML/AI model in the first place is to extract insights that can be used to solve problems, make decisions etc. If you don't test your naive bayes or decision tree model with test data, you won't know if the information that are given to you by those model mean anything. They may not even help you solve problems or give you relevant information. Yes they may spout out big numbers and classify things, but are those result what you're looking for? Are the result relevant to your problem? Can the result be used to solve what you are trying to do?</p>
  
  <p>Using test data gives you the opportunity to see if your model gives you the best insights and the best solution to your problems. So here are the takeaways from my answer:</p>
  
  <ul>
  <li>Test data can be used to test your model against unseen data</li>
  <li>You can gain score (evaluation) on your model when you test it with test data, which in turn can be used to fine tune your model</li>
  <li>You can see if the answer the model gives you is any relevant to your initial problem. It its not, then it may be best to use some other algorithm.</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-21T07:43:41.283Z
Answers.LastActivityDate: 2020-04-21T07:43:41.283Z
Answers.UserDisplayName: Firdaus Indradhirmaya
Answers.Body: <p>In the field of ML and AI, you should always remember that before choosing any algorithm you should know the data .One should always start with Data Analyzing, which itself is field for this critical job. Decision Tree can never work with its best optimization without tuning the datasets. Here is a great article  that you can refer : <a href="https://towardsdatascience.com/how-to-tune-a-decision-tree-f03721801680" rel="nofollow noreferrer">Tuning Decision Tree</a></p>
  
  <p>Purpose of test data in naive byes classifier:
  1) It's necessity to check the accuracy, hits, hit rates, coverage, diversity, novelty, etc. metrics.</p>
  
  <p>2) It hypertune your testdata(as anti_train_set) also by using mean, standard deviation, variance.</p>
  
  <p>I really think that you should try other algorithms to train your datasets. I can't name all of them. However, in Neural network, rnn, cnn, rbm are some great algorithm to work with.</p>
  
  <p>Please always remember that Machine learning is like an art, where datasets (test, train, evaluated) are colors and its up-to us to use the right amount of them.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-07T17:38:44.713Z
Answers.LastActivityDate: 2020-04-07T17:38:44.713Z
Answers.UserDisplayName: techPirate99
Answers.Body: <p>When we train a model using a data train, sometimes the resulting score is very high. This makes us believe that our model is very good. But when predicting actual data, the resulting score is very low. Why?? This means that the trained model is <strong>overfit</strong> (to data train) and fail to predict anything useful on yet-<strong>unseen</strong> data.</p>
  
  <p>That's why we have to check our model to test data (predict test data) and compare the accuracy between data train and data test. If the accuracy is <strong>not too far away</strong>, then our model does <strong>not overfit</strong>.</p>
  
  <p>Later, we can improve our model with <strong>Cross Validation</strong> (<a href="https://scikit-learn.org/stable/modules/cross_validation.html#cross-validation" rel="nofollow noreferrer">Reference</a>) that split our data train to n-split data and take one of that to became data train. Then we take the average of Cross Validation score.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-21T09:24:09.193Z
Answers.LastActivityDate: 2020-04-21T09:24:09.193Z
Answers.UserDisplayName: Iriyanto
Body: <p>What is the benefit of a test data set, especially for naive bayes estimator or decision tree construction?</p>
  
  <p>When using a naive bayes classifier the probabilities are a fact. As far as I know there is nothing one could tune (like the weights in a neural net). So what is the purpose of the test data set? Simply to know if one can apply naive bayes or not?</p>
  
  <p>Similiarly what is the benefit of the test data set when constructing a decision tree. We alread use the gini impurity to construct the best possibe decision tree and there is nothing we could do when we get bad results with the test data set.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-07T16:34:56.477Z
Comments.Text: I think that this answer can apply also to your question https://ai.stackexchange.com/questions/20034/why-do-we-need-both-the-validation-set-and-test-set/20049#20049
Comments.UserDisplayName: Edoardo Guerriero
Tags: <datasets><decision-trees><naive-bayes>
Title: What is the meaning of test data set in naive bayes classifier or decision trees?
UserDisplayName: MScott
ThreadText: # What is the meaning of test data set in naive bayes classifier or decision trees?
  
  (Asked by: username_0 on 2020-04-07)
  
  What is the benefit of a test data set, especially for naive bayes estimator or decision tree construction?
  
  When using a naive bayes classifier the probabilities are a fact. As far as I know there is nothing one could tune (like the weights in a neural net). So what is the purpose of the test data set? Simply to know if one can apply naive bayes or not?
  
  Similiarly what is the benefit of the test data set when constructing a decision tree. We alread use the gini impurity to construct the best possibe decision tree and there is nothing we could do when we get bad results with the test data set.
  
  **Tags:** datasets, decision-trees, naive-bayes
  > **Comments:**
  > **username_1**: I think that this answer can apply also to your question https://ai.stackexchange.com/questions/20034/why-do-we-need-both-the-validation-set-and-test-set/20049#20049  
  
  ---
  
  ## 6 answers
  
  ### Answer by username_2 on 2020-04-21. Score: 1
  
  In machine learning, we can use all the datasets as training data in a model. But if there are too many data sets, or too much data, and we do not split them up, our model may be not produce acceptable results.
  
  Why?
  
  Because if the model studies too much training data, it may be [overfitted](https://en.wikipedia.org/wiki/Overfitting).
  
  (Just like when you cram for a test, and get overloaded with too much information!)
  
  What I mean is, your model is only familiar with the data you provide, not for the new data.
  
  So we need to use test data to train our algorithm. Naive Bayes and Decision Tree Classifier are no exception because they can produce an overfitted model based on train data.
  
  So we test it on the data test to know how well the method works in relation to the problem.
  
  > Most data scientists divide their data (with answers, that is historical data) into three portions: training data, cross-validation data and testing data. The training data is used to make sure the machine recognizes patterns in the data, the cross-validation data is used to ensure better accuracy and efficiency of the algorithm used to train the machine, and the test data is used to see how well the machine can predict new answers based on its training.
  
  SOURCE: <https://www.researchgate.net/post/What_is_training_and_testing_data_in_machine_learning>
  
  
  ---
  
  ### Answer by username_3 on 2020-04-07. Score: 1
  
  Your assumption about the test data is not correct completely. Maybe you use the test data to tune your learning algorithm to work better on the test data, but it's not the whole thing. Sometimes you need to know that the ML method is working or not and have a sense about how much does it work!
  
  You have other scenarios that you want to evaluate your method:
  
  1. Compare the result of the leaner with other techniques. For example, you are considering DT versus an SVM classifier over a data set. If you want to compare them, you need a value to found such a sense about the comparison.
  2. Sometimes you are using an ensemble method and you want to tune some parameters to balance between using different ML methods. Hence, you need to evaluate these learning methods (DT, Naive Bayes) to improve the ensemble method.
  
  
  ---
  
  ### Answer by username_4 on 2020-04-21. Score: 0
  
  One way to test the accuracy of trained model is by testing it using data test. By testing we are able to check the accuracy. Whether your model is a good model or not depends heavily on the accuracy, if your accuracy is too low or too high (eg. up to 99%~100%), there could be some problem in your model.
  
  For further information on the example in data test, you can access <https://jakevdp.github.io/PythonDataScienceHandbook/05.05-naive-bayes.html>
  
  Hope this helps
  
  
  ---
  
  ### Answer by username_5 on 2020-04-21. Score: 0
  
  I actually pondered this question a few months ago, so i understand your point of view!
  
  You are correct in assuming that if you already build your tree or calculate your probabilities, what is the point of using test data? Because your model is fixed the way they are, no matter if you use test data or not. Well, the purpose of test data is not only to test your model against unseen data and get some evaluation score. But it is also to test if your model is the right fit for your problem.
  
  One of the main reason why we all build ML/AI model in the first place is to extract insights that can be used to solve problems, make decisions etc. If you don't test your naive bayes or decision tree model with test data, you won't know if the information that are given to you by those model mean anything. They may not even help you solve problems or give you relevant information. Yes they may spout out big numbers and classify things, but are those result what you're looking for? Are the result relevant to your problem? Can the result be used to solve what you are trying to do?
  
  Using test data gives you the opportunity to see if your model gives you the best insights and the best solution to your problems. So here are the takeaways from my answer:
  
  * Test data can be used to test your model against unseen data
  * You can gain score (evaluation) on your model when you test it with test data, which in turn can be used to fine tune your model
  * You can see if the answer the model gives you is any relevant to your initial problem. It its not, then it may be best to use some other algorithm.
  
  
  ---
  
  ### Answer by username_6 on 2020-04-07. Score: 0
  
  In the field of ML and AI, you should always remember that before choosing any algorithm you should know the data .One should always start with Data Analyzing, which itself is field for this critical job. Decision Tree can never work with its best optimization without tuning the datasets. Here is a great article that you can refer : [Tuning Decision Tree](https://towardsdatascience.com/how-to-tune-a-decision-tree-f03721801680)
  
  Purpose of test data in naive byes classifier:
  1) It's necessity to check the accuracy, hits, hit rates, coverage, diversity, novelty, etc. metrics.
  
  2) It hypertune your testdata(as anti\_train\_set) also by using mean, standard deviation, variance.
  
  I really think that you should try other algorithms to train your datasets. I can't name all of them. However, in Neural network, rnn, cnn, rbm are some great algorithm to work with.
  
  Please always remember that Machine learning is like an art, where datasets (test, train, evaluated) are colors and its up-to us to use the right amount of them.
  
  
  ---
  
  ### Answer by username_7 on 2020-04-21. Score: 0
  
  When we train a model using a data train, sometimes the resulting score is very high. This makes us believe that our model is very good. But when predicting actual data, the resulting score is very low. Why?? This means that the trained model is **overfit** (to data train) and fail to predict anything useful on yet-**unseen** data.
  
  That's why we have to check our model to test data (predict test data) and compare the accuracy between data train and data test. If the accuracy is **not too far away**, then our model does **not overfit**.
  
  Later, we can improve our model with **Cross Validation** ([Reference](https://scikit-learn.org/stable/modules/cross_validation.html#cross-validation)) that split our data train to n-split data and take one of that to became data train. Then we take the average of Cross Validation score.
  
  
  ---
  
  