Answers.Body: <p>You can't label things you don't know.  The goal of labeling is to label the things you want the classifier to learn so that when you run it in inference mode you can discover what is in your data (new data that you didn't use for training, validating, or testing).</p>
  
  <p>It is not a good idea to label small objects like the 'blue water' unless it is important to you to discover these fine details in inference mode.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-12T11:00:39.390Z
Answers.Text: Thank you for your answer. Can I ask if you previously worked on multi-label classification and what metrics you used if you did?
Answers.UserDisplayName: user
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-11T14:38:34.693Z
Answers.LastActivityDate: 2020-01-11T14:38:34.693Z
Answers.UserDisplayName: Brian O'Donnell
Answers.Body: <p>I think what you are actually talking about is semantic segmentation (where you label pixels individually).</p>
  <p>There is a difference in theses tasks like Classification, Detection or Semantic Segmentation.</p>
  <p>Classification refers to the task of giving a (usually) single label to the whole image, e.g. cat. But as you already noticed this does not necessarily end in a clear labeling policy, since you basically always have multiple classes in one image. However, a ANN usually learns the most relevant (biggest, nearest) object in an image to set it to the corresponding class (but this of course again depends on how the images are labeled). At inferencing of the ANN, you then get a probability distribution over all predefined classes. You can, of course, use this to take the K most relevant classes instead of just the single most relevant classes, to cover cases where multiple objects are probably present. However common output layers, e.g. like Softmax, are usually designed to favour for one single class instead of multiple classes, so you should keep that in mind or consider using a better suited output layer function for your use-case.</p>
  <p>To have a more general approach in the task of object detection you classify multiple objects in an image (usually as bounding boxes). That means you label all predefined objects with their position and class in an image.</p>
  <p>And the most general approach here would be to do semantic segmentation that labels every pixel to a corresponding class, which then gives you the actual object borders etc. You can also label pixels as &quot;voids&quot; (or something like that) to cover unknown classes or classes that are not considered in your dataset. However creating such a dataset is a horrible amount of work.</p>
  <p>To clear this up: look at your actual use-case, and think about what you actually want your neural network to do. According to this, you then think of a labeling policy and label your data.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-12T11:05:56.833Z
Answers.Text: Thank you. I originally wanted to do semantic segmentation. However, I don't know many of the objects in the picture and I also would need a lot of time to prepare the data. So I felt like multi-label was a decent approach to consider.
Answers.UserDisplayName: user
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-15T21:13:38.827Z
Answers.Text: Okay. So for multi-label I think its best to use a normal sigmoid function on your final output layer and then threshold on the class outputs. The typical used softmax layer at output layer in general classification tasks tries to favour one class due to the normalization over all possible classes. So this might lead to unwanted behaviours.
Answers.UserDisplayName: Marcel_marcel1991
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-11T15:26:45.380Z
Answers.LastActivityDate: 2021-06-08T11:02:33.200Z
Answers.LastEditDate: 2021-06-08T11:02:33.200Z
Answers.UserDisplayName: Marcel_marcel1991
Answers.Body: <p>I would classify each pixel separately instead of giving a label to the whole image. Sadly preparing the training data is very tedious and time-consuming.</p>
  
  <p>Let's say the input image has dimensions of 200 x 300 x 3 (RGB) and there are two classes of regions you want to identify. A few approaches come to mind:</p>
  
  <p>1) Train two separate networks, each forecasting a binary mask of size 200 x 300 of the object class in guestion.</p>
  
  <p>2) Train a single network with a binary output of size 200 x 300 x 2 (sigmoid activation)</p>
  
  <p>3) Train a single network with a binary output of size 200 x 300 x 3 (softmax activation), the 3rd class is for "other"</p>
  
  <p>If you are uncertain of some regions you can leave its class probability to 50% and it won't affect cross-entropy losses.</p>
  
  <p>Option 1 is easiest to get started with but training a single network should be more computationally more efficient than training two separate ones. In addition options 1 and 2 can forecast a single pixel belonging to both classes with 100% probability unlike the network of option 3.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-14T18:00:39.633Z
Answers.LastActivityDate: 2020-01-14T18:00:39.633Z
Answers.UserDisplayName: NikoNyrh
Body: <p>I have images that contain lots of elements. Some I know, some I don't. I want to know if it's ok to only label those I do know. Let's take this image for example. I would label the green stuff and the worm but leave the rest unlabeled.
  Is that ok?</p>
  
  <p>Another question I would also like to ask is how concise I should be in labeling. For instance, You can see in the picture a bit of blue behind the green plant. So should I label that bit and say water or leave it unlabeled?</p>
  
  <p><strong>EDIT:</strong></p>
  
  <p>I also want to ask if it's ok to label only the things I'm interested in even if they take <strong>up to</strong>  30% of the picture? Won't the neural network be confused by all the details in the picture that it perceives and that I label as A for example, even if A is just a part of it?</p>
  
  <p>Another question would be, let's say I have labels A, B and C. I have an image in which I'm a bit confused if a certain object is of label B <strong>or</strong> A <strong>or</strong> even a totally different class other than (A,B,C). What should I do in this instance?</p>
  
  <p><em>I'm having a hard time with the dataset. It would take an expert to label this correctly. But I want to do things as cleanly as possible, so all the effort doesn't go to waste. I would really appreciate your help. Thank you guys.</em> </p>
  
  <p><a href="https://i.sstatic.net/37OdN.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/37OdN.jpg" alt="enter image description here"></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-11T11:21:57.880Z
Comments.Text: Are you doing classification or object detection?
Comments.UserDisplayName: Sharan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-11T11:57:46.510Z
Comments.Text: @Sharan Hi I'm doing classification
Comments.UserDisplayName: user
Tags: <classification><datasets>
Title: How to correctly label images for multi-label classification?
UserDisplayName: user
ThreadText: # How to correctly label images for multi-label classification?
  
  (Asked by: username_0 on 2020-01-11)
  
  I have images that contain lots of elements. Some I know, some I don't. I want to know if it's ok to only label those I do know. Let's take this image for example. I would label the green stuff and the worm but leave the rest unlabeled.
  Is that ok?
  
  Another question I would also like to ask is how concise I should be in labeling. For instance, You can see in the picture a bit of blue behind the green plant. So should I label that bit and say water or leave it unlabeled?
  
  **EDIT:**
  
  I also want to ask if it's ok to label only the things I'm interested in even if they take **up to** 30% of the picture? Won't the neural network be confused by all the details in the picture that it perceives and that I label as A for example, even if A is just a part of it?
  
  Another question would be, let's say I have labels A, B and C. I have an image in which I'm a bit confused if a certain object is of label B **or** A **or** even a totally different class other than (A,B,C). What should I do in this instance?
  
  *I'm having a hard time with the dataset. It would take an expert to label this correctly. But I want to do things as cleanly as possible, so all the effort doesn't go to waste. I would really appreciate your help. Thank you guys.*
  
  [![enter image description here](https://i.sstatic.net/37OdN.jpg)](https://i.sstatic.net/37OdN.jpg)
  
  **Tags:** classification, datasets
  > **Comments:**
  > **username_1**: Are you doing classification or object detection?  
  > **username_0**: @Sharan Hi I'm doing classification  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_2 on 2020-01-11. Score: 2
  
  You can't label things you don't know. The goal of labeling is to label the things you want the classifier to learn so that when you run it in inference mode you can discover what is in your data (new data that you didn't use for training, validating, or testing).
  
  It is not a good idea to label small objects like the 'blue water' unless it is important to you to discover these fine details in inference mode.
  
  > **Comments:**
  > **username_0**: Thank you for your answer. Can I ask if you previously worked on multi-label classification and what metrics you used if you did?  
  
  ---
  
  ### Answer by username_3 on 2020-01-11. Score: 1
  
  I think what you are actually talking about is semantic segmentation (where you label pixels individually).
  
  There is a difference in theses tasks like Classification, Detection or Semantic Segmentation.
  
  Classification refers to the task of giving a (usually) single label to the whole image, e.g. cat. But as you already noticed this does not necessarily end in a clear labeling policy, since you basically always have multiple classes in one image. However, a ANN usually learns the most relevant (biggest, nearest) object in an image to set it to the corresponding class (but this of course again depends on how the images are labeled). At inferencing of the ANN, you then get a probability distribution over all predefined classes. You can, of course, use this to take the K most relevant classes instead of just the single most relevant classes, to cover cases where multiple objects are probably present. However common output layers, e.g. like Softmax, are usually designed to favour for one single class instead of multiple classes, so you should keep that in mind or consider using a better suited output layer function for your use-case.
  
  To have a more general approach in the task of object detection you classify multiple objects in an image (usually as bounding boxes). That means you label all predefined objects with their position and class in an image.
  
  And the most general approach here would be to do semantic segmentation that labels every pixel to a corresponding class, which then gives you the actual object borders etc. You can also label pixels as "voids" (or something like that) to cover unknown classes or classes that are not considered in your dataset. However creating such a dataset is a horrible amount of work.
  
  To clear this up: look at your actual use-case, and think about what you actually want your neural network to do. According to this, you then think of a labeling policy and label your data.
  
  > **Comments:**
  > **username_0**: Thank you. I originally wanted to do semantic segmentation. However, I don't know many of the objects in the picture and I also would need a lot of time to prepare the data. So I felt like multi-label was a decent approach to consider.  
  > **username_3**: Okay. So for multi-label I think its best to use a normal sigmoid function on your final output layer and then threshold on the class outputs. The typical used softmax layer at output layer in general classification tasks tries to favour one class due to the normalization over all possible classes. So this might lead to unwanted behaviours.  
  
  ---
  
  ### Answer by username_4 on 2020-01-14. Score: 0
  
  I would classify each pixel separately instead of giving a label to the whole image. Sadly preparing the training data is very tedious and time-consuming.
  
  Let's say the input image has dimensions of 200 x 300 x 3 (RGB) and there are two classes of regions you want to identify. A few approaches come to mind:
  
  1) Train two separate networks, each forecasting a binary mask of size 200 x 300 of the object class in guestion.
  
  2) Train a single network with a binary output of size 200 x 300 x 2 (sigmoid activation)
  
  3) Train a single network with a binary output of size 200 x 300 x 3 (softmax activation), the 3rd class is for "other"
  
  If you are uncertain of some regions you can leave its class probability to 50% and it won't affect cross-entropy losses.
  
  Option 1 is easiest to get started with but training a single network should be more computationally more efficient than training two separate ones. In addition options 1 and 2 can forecast a single pixel belonging to both classes with 100% probability unlike the network of option 3.
  
  
  ---
  
  