Answers.Body: <p>This may not seem trivial but <strong>yes</strong>, the models we train <em>can</em> potentially learn a variety of things they weren't intended to learn. There are already some examples in computer vision. A typical convolutional network learns things like edge detection, various potentially useful masks etc. in the early layers while learns more high-level features like eyes, nose etc. in higher layers.</p>
  
  <p>It is reasonable too. Given the dataset size is moderately high and the model is trained for long enough, a sufficiently deep network learns various kinds of hidden representations, <em>which may not even be specific to the task at hand</em>. This is the reason transfer learning works very well even on a host of different datasets.</p>
  
  <p>This is limited since not all the learnable things can be described using mathematics. So, the answer is a surprising no. The model does learn some extra things other than the task at hand.</p>
  
  <p><strong>P.S.</strong>: There was also a case when a group of researchers trained a model to make a robot walk. It turned out the robot had learned to recognize faces too and reacted in different ways on seeing different faces. I saw the video on YouTube a while ago and couldn't find the exact video to post the link here, anyways.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T10:51:47.850Z
Answers.Text: Good answer, thanks. But for simple regression, say, there is nothing new unless we label new data, is that right?
Answers.UserDisplayName: schoon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T11:23:45.533Z
Answers.Text: Yeah @schoon, I can't think of any way a simple regression model can learn something interesting. But, since the structure of data in classification and regression problems is more or less the same, it may still learn something which might not be perceivable by us. This gave me an idea though, we can try to do transfer learning in case of regression tasks and see if it yields any useful results. If it does, we can say that the model has indeed learned something else other than the task at hand!
Answers.UserDisplayName: Suyog Jadhav
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T10:15:27.190Z
Answers.LastActivityDate: 2018-07-10T10:27:04.300Z
Answers.LastEditDate: 2018-07-10T10:27:04.300Z
Answers.UserDisplayName: Suyog Jadhav
Body: <p>If I do supervised learning the model learns from the labeled input data. This seems to be quite often a small set of human annotated data. </p>
  
  <p>Is it true to say this is the only 'learning' the model does? </p>
  
  <p>It seems like the small data set has a huge influence on the model. Can it be made better using future unlabeled data? </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-07-10T11:41:07.200Z
Comments.Text: Could you please explain what you mean saying that the labeled data is quite often a small set of human annotated data? Do you mean that the inputs are manually labeled?
Comments.UserDisplayName: gvgramazio
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-07-10T11:42:20.180Z
Comments.Text: Also, without labels, how could you expect to train a NN? You cannot provide a cost function for these data.
Comments.UserDisplayName: gvgramazio
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-07-11T10:44:28.727Z
Comments.Text: @gvgramazio yes manual labeling of a sample.
Comments.UserDisplayName: schoon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-07-11T12:49:05.920Z
Comments.Text: Then if you use supervised learning you cannot use those labeled data. Instead, if you use unsupervised learning you could use unlabeled data.
Comments.UserDisplayName: gvgramazio
Tags: <machine-learning>
Title: Does machine learning continue to learn?
UserDisplayName: schoon
ThreadText: # Does machine learning continue to learn?
  
  (Asked by: username_0 on 2018-07-10)
  
  If I do supervised learning the model learns from the labeled input data. This seems to be quite often a small set of human annotated data.
  
  Is it true to say this is the only 'learning' the model does?
  
  It seems like the small data set has a huge influence on the model. Can it be made better using future unlabeled data?
  
  **Tags:** machine-learning
  > **Comments:**
  > **username_1**: Could you please explain what you mean saying that the labeled data is quite often a small set of human annotated data? Do you mean that the inputs are manually labeled?  
  > **username_1**: Also, without labels, how could you expect to train a NN? You cannot provide a cost function for these data.  
  > **username_0**: @gvgramazio yes manual labeling of a sample.  
  > **username_1**: Then if you use supervised learning you cannot use those labeled data. Instead, if you use unsupervised learning you could use unlabeled data.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2018-07-10. Score: 1 | Accepted answer
  
  This may not seem trivial but **yes**, the models we train *can* potentially learn a variety of things they weren't intended to learn. There are already some examples in computer vision. A typical convolutional network learns things like edge detection, various potentially useful masks etc. in the early layers while learns more high-level features like eyes, nose etc. in higher layers.
  
  It is reasonable too. Given the dataset size is moderately high and the model is trained for long enough, a sufficiently deep network learns various kinds of hidden representations, *which may not even be specific to the task at hand*. This is the reason transfer learning works very well even on a host of different datasets.
  
  This is limited since not all the learnable things can be described using mathematics. So, the answer is a surprising no. The model does learn some extra things other than the task at hand.
  
  **P.S.**: There was also a case when a group of researchers trained a model to make a robot walk. It turned out the robot had learned to recognize faces too and reacted in different ways on seeing different faces. I saw the video on YouTube a while ago and couldn't find the exact video to post the link here, anyways.
  
  > **Comments:**
  > **username_0**: Good answer, thanks. But for simple regression, say, there is nothing new unless we label new data, is that right?  
  > **username_2**: Yeah @schoon, I can't think of any way a simple regression model can learn something interesting. But, since the structure of data in classification and regression problems is more or less the same, it may still learn something which might not be perceivable by us. This gave me an idea though, we can try to do transfer learning in case of regression tasks and see if it yields any useful results. If it does, we can say that the model has indeed learned something else other than the task at hand!  
  
  ---
  
  