Answers.Body: <p>To my knowledge the deployment model (that you will test on underwater images) as inference will not have a negative effect. Yet drawings may even help differentiate some classes at training and inference. Provided that you won't use drawings in inference, adding them in training phase will not necessarily hurt the accuracy. Note that a drawing of a particular class should not be in the search domains of other classes, namely, a drawing of a particular class should not be the same with the other classes.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-10T02:09:37.370Z
Answers.Text: Though if I were to do it I would use the drawings for pretraining the weights, if the drawings data is big enough and if the drawings are about underwater images as well.
Answers.UserDisplayName: Clement
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-10T07:50:36.223Z
Answers.Text: @voo_doo I don't quite understand this bit: " a drawing of a particular class should not be the same with the other classes".
Answers.UserDisplayName: user
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-10T09:26:07.587Z
Answers.Text: @bookfreak the author is saying that there are drawings of a certain class (say, a particular sea-grass). That drawing should not be similar in look to other real underwater images of other classes. If that makes sense...
Answers.UserDisplayName: bit_scientist
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-10T09:35:36.090Z
Answers.Text: @voo_doo Do you have a link? It would be better to read the whole thing.
Answers.UserDisplayName: user
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-10T09:40:57.787Z
Answers.Text: @bookfreak I guess you have the data set, post some of the drawings and other underwater images of other classes here as an edit.
Answers.UserDisplayName: bit_scientist
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-10T00:46:20.683Z
Answers.LastActivityDate: 2020-01-10T00:46:20.683Z
Answers.UserDisplayName: bit_scientist
Answers.Body: <p>In general, <em>training on any distribution other that what you are testing on could give worse performance</em>. The model learns to fit the distribution you train it on.</p>
  <p>Certainly more data of a sufficiently close distribution will help, e.g. training on Imagenet and testing on CIFAR, but hand-drawings vs natural images seem very different and unless that is what you're wanting to test on, I would imagine it will hurt performance. Sufficiently similar data might include augmentations of the original data, such as crops, flips, blurring etc, as commonly used to effectively enlarge the training set. This is ok as this extra data is effectively &quot;in distribution&quot;.</p>
  <p>Think of it like this: you are training the model, of finite capacity, to learn your intended distribution plus some info about the extra data that it will never see again, so part of it's modelling capacity is being used up on a pointless task.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-01T08:56:14.320Z
Answers.LastActivityDate: 2023-08-01T08:56:14.320Z
Answers.UserDisplayName: Carl
Body: <p>I'm training a multi-label classifier that's supposed to be tested on underwater images. I'm wondering if feeding the model drawings of a certain class plus real images can affect the results badly. Was there a study on this? Or are there any past experiences anyone could share to help?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-10T14:16:32.077Z
Comments.Text: Intuitively, to minimize the biases introduced by the drawings you may want to have your drawings equally distributed among all the different classes that you have. Otherwise the system is likely to focus on the particular traits that constitute a drawing to predict the classes. Maybe applying some kind of style transfer as a preprocessing step to render all your input data alike could be an interesting idea to explore.
Comments.UserDisplayName: Jeanba
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-11T04:46:15.083Z
Comments.Text: I think if the drawings are realistic drawings they maybe used for weight pretraining before actual training if the data size is large enough. If the drawings are not realistic (for example maybe some impressionism drawings) then it should not be used. I don't think the drawings should be trained along side the real images as it may disrupt the accuracy and training process
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-11T08:11:32.410Z
Comments.Text: @ClementHui When you say weight pretraining, do you mean that I should unfreeze all layers?And by "data size large enough" how much is large enough?
Comments.UserDisplayName: user
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-11T08:17:39.553Z
Comments.Text: When doing image detection, you should try using one of the available models like ResNet with imageNet weights. Then try training the imageNet model on the drawings but only unfreeze the last few layers. Then you should train on the main dataset.
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-11T08:18:04.233Z
Comments.Text: By large enough I mean like 1000/class or more
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-11T08:18:27.920Z
Comments.Text: Btw how large is your main dataset size?
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-11T08:24:44.390Z
Comments.Text: @ClementHui I'm still working on the dataset, but I have 5000 images total for 20 labels.
Comments.UserDisplayName: user
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-11T08:27:13.037Z
Comments.Text: @ClementHui Is 1000/class with/without data augmentation?
Comments.UserDisplayName: user
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-11T08:29:27.850Z
Comments.Text: @ClementHui I should also probably mention that it's a multi-label classification. That probably has totally different requirements
Comments.UserDisplayName: user
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-30T02:34:30.313Z
Comments.Text: I'm curious to know the results of your efforts, here are a few studies that appear to be in the realm of your need.
  [first academic link](https://www.researchgate.net/publication/337922291_Analysis_and_Recognition_of_Hand-Drawn_Images_with_Effective_Data_Handling)
  [second academic link](https://www.researchgate.net/publication/306081553_Classification_of_Photo_and_Sketch_Images_Using_Convolutional_Neural_Networks)
  [Journal link](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0183838)
Comments.UserDisplayName: CoffeeBaconAddict
Tags: <deep-learning><classification>
Title: Can training a model on a dataset composed by real images and drawings hurt the training process of a real-world application model?
UserDisplayName: user
ThreadText: # Can training a model on a dataset composed by real images and drawings hurt the training process of a real-world application model?
  
  (Asked by: username_0 on 2020-01-09)
  
  I'm training a multi-label classifier that's supposed to be tested on underwater images. I'm wondering if feeding the model drawings of a certain class plus real images can affect the results badly. Was there a study on this? Or are there any past experiences anyone could share to help?
  
  **Tags:** deep-learning, classification
  > **Comments:**
  > **username_1**: Intuitively, to minimize the biases introduced by the drawings you may want to have your drawings equally distributed among all the different classes that you have. Otherwise the system is likely to focus on the particular traits that constitute a drawing to predict the classes. Maybe applying some kind of style transfer as a preprocessing step to render all your input data alike could be an interesting idea to explore.  
  > **username_2**: I think if the drawings are realistic drawings they maybe used for weight pretraining before actual training if the data size is large enough. If the drawings are not realistic (for example maybe some impressionism drawings) then it should not be used. I don't think the drawings should be trained along side the real images as it may disrupt the accuracy and training process  
  > **username_0**: @ClementHui When you say weight pretraining, do you mean that I should unfreeze all layers?And by "data size large enough" how much is large enough?  
  > **username_2**: When doing image detection, you should try using one of the available models like ResNet with imageNet weights. Then try training the imageNet model on the drawings but only unfreeze the last few layers. Then you should train on the main dataset.  
  > **username_2**: By large enough I mean like 1000/class or more  
  > **username_2**: Btw how large is your main dataset size?  
  > **username_0**: @ClementHui I'm still working on the dataset, but I have 5000 images total for 20 labels.  
  > **username_0**: @ClementHui Is 1000/class with/without data augmentation?  
  > **username_0**: @ClementHui I should also probably mention that it's a multi-label classification. That probably has totally different requirements  
  > **username_3**: I'm curious to know the results of your efforts, here are a few studies that appear to be in the realm of your need. [first academic link](https://www.researchgate.net/publication/337922291_Analysis_and_Recognition_of_Hand-Drawn_Images_with_Effective_Data_Handling) [second academic link](https://www.researchgate.net/publication/306081553_Classification_of_Photo_and_Sketch_Images_Using_Convolutional_Neural_Networks) [Journal link](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0183838)  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2020-01-10. Score: 0
  
  To my knowledge the deployment model (that you will test on underwater images) as inference will not have a negative effect. Yet drawings may even help differentiate some classes at training and inference. Provided that you won't use drawings in inference, adding them in training phase will not necessarily hurt the accuracy. Note that a drawing of a particular class should not be in the search domains of other classes, namely, a drawing of a particular class should not be the same with the other classes.
  
  > **Comments:**
  > **username_2**: Though if I were to do it I would use the drawings for pretraining the weights, if the drawings data is big enough and if the drawings are about underwater images as well.  
  > **username_0**: @voo_doo I don't quite understand this bit: " a drawing of a particular class should not be the same with the other classes".  
  > **username_4**: @bookfreak the author is saying that there are drawings of a certain class (say, a particular sea-grass). That drawing should not be similar in look to other real underwater images of other classes. If that makes sense...  
  > **username_0**: @voo_doo Do you have a link? It would be better to read the whole thing.  
  > **username_4**: @bookfreak I guess you have the data set, post some of the drawings and other underwater images of other classes here as an edit.  
  
  ---
  
  ### Answer by username_5 on 2023-08-01. Score: 0
  
  In general, *training on any distribution other that what you are testing on could give worse performance*. The model learns to fit the distribution you train it on.
  
  Certainly more data of a sufficiently close distribution will help, e.g. training on Imagenet and testing on CIFAR, but hand-drawings vs natural images seem very different and unless that is what you're wanting to test on, I would imagine it will hurt performance. Sufficiently similar data might include augmentations of the original data, such as crops, flips, blurring etc, as commonly used to effectively enlarge the training set. This is ok as this extra data is effectively "in distribution".
  
  Think of it like this: you are training the model, of finite capacity, to learn your intended distribution plus some info about the extra data that it will never see again, so part of it's modelling capacity is being used up on a pointless task.
  
  
  ---
  
  