Answers.Body: <p>You should, absolutely.</p>
  <p>Even more, if you have prior knowledge regarding the distribution of the disease (like 1 case every 1000) I would strive to get a training dataset close to that distribution, even if it means collecting and adding more control cases with empty masks.</p>
  <p>At the end you'll probably end up with a really imbalanced dataset, the way to tackle this issue is using the <a href="https://arxiv.org/abs/1708.02002" rel="nofollow noreferrer">focal loss</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-12T10:44:48.603Z
Answers.Text: Thank you for your response. But I tried this approach earlier and used the focal loss, but the result was unsatisfactory. I have used model  unet-resnet34. The tumor patch is tiny, and you can say it is a small dot on 512X512 images, which is hard to segment. Therefore, I'm using only mask pictures. And at this time stamp, I'm getting an IOU of 74%. Once the training is complete, I'll freeze all the layers except the last one and feed mixed images which are empty masks and non-empty masks. What do you say abo this approach?
Answers.UserDisplayName: kal1619
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-10T21:58:17.803Z
Answers.LastActivityDate: 2022-10-10T21:58:17.803Z
Answers.UserDisplayName: Edoardo Guerriero
Answers.Body: <p>I share with you my experience with TensorFlow 2.0 (TF2.0).  I don't know for other deep learning packages.  In TF2.0, you must pass <code>x_train</code> and <code>y_train</code> together during training, where <code>x_train</code> is the image and <code>y_train</code> is the mask.  (Similar holds for <code>x_test</code> and <code>y_test</code>, which must be passed together during evaluation.)  This means that you must have a mask for both images with tumor and images without tumor.  In your case, <code>y_train</code> would be a binary mask, since you have two pixel classes (tumor and no tumor).  For images with no tumor, the mask would be all <code>0's</code> (what you call an &quot;empty mask&quot;).  For images with tumor, the mask would be <code>0's</code> everywhere but <code>1's</code> where there is tumor (what you call the &quot;patch&quot;).  Note that the masks are essentially images (or numpy arrays, if you prefer) of the same size as your medical images, except that the mask has binary pixel values according to tumor present (<code>1's</code>) or absent (<code>0's</code>).</p>
  <p>Thus, the answer to your question, <em>&quot;Should I also feed my model empty masks along with patches (non empty mask)?&quot;</em>, is <code>Yes</code>.  In fact, in TF2.0, you absolutely must. (A separate post can be started if you have questions about model performance.)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-10T12:43:38.003Z
Answers.LastActivityDate: 2022-11-10T14:56:16.320Z
Answers.LastEditDate: 2022-11-10T14:56:16.320Z
Answers.UserDisplayName: Snehal Patel
Body: <p>Total Dataset :- 100 (on case level)</p>
  <p>Training :- 76 cases (18000 slices)
  Validation :- 19 cases (4000 slices)
  Test  :- 5 cases (2000 slices)</p>
  <p>I have a dataset that consists of approx. Eighteen thousand images, out of which approx. Fifteen thousand images are of the normal patient and around 3000 images of patients having some diseases. Now, for these 18000 images, I also have their segmentation mask. So, 15000 segmentations masks are empty, and 3000 have patches.</p>
  <p>Should I also feed my model (deep learning, i.e., unet with resnet34 backbone) empty masks along with patches (non empty mask)?</p>
  
Tags: <image-segmentation>
Title: Are masks needed for images that don't contain the object of interest in (binary) Image Segmentation tasks?
UserDisplayName: kal1619
ThreadText: # Are masks needed for images that don't contain the object of interest in (binary) Image Segmentation tasks?
  
  (Asked by: username_0 on 2022-10-10)
  
  Total Dataset :- 100 (on case level)
  
  Training :- 76 cases (18000 slices)
  Validation :- 19 cases (4000 slices)
  Test :- 5 cases (2000 slices)
  
  I have a dataset that consists of approx. Eighteen thousand images, out of which approx. Fifteen thousand images are of the normal patient and around 3000 images of patients having some diseases. Now, for these 18000 images, I also have their segmentation mask. So, 15000 segmentations masks are empty, and 3000 have patches.
  
  Should I also feed my model (deep learning, i.e., unet with resnet34 backbone) empty masks along with patches (non empty mask)?
  
  **Tags:** image-segmentation
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-10-10. Score: 0
  
  You should, absolutely.
  
  Even more, if you have prior knowledge regarding the distribution of the disease (like 1 case every 1000) I would strive to get a training dataset close to that distribution, even if it means collecting and adding more control cases with empty masks.
  
  At the end you'll probably end up with a really imbalanced dataset, the way to tackle this issue is using the [focal loss](https://arxiv.org/abs/1708.02002).
  
  > **Comments:**
  > **username_0**: Thank you for your response. But I tried this approach earlier and used the focal loss, but the result was unsatisfactory. I have used model  unet-resnet34. The tumor patch is tiny, and you can say it is a small dot on 512X512 images, which is hard to segment. Therefore, I'm using only mask pictures. And at this time stamp, I'm getting an IOU of 74%. Once the training is complete, I'll freeze all the layers except the last one and feed mixed images which are empty masks and non-empty masks. What do you say abo this approach?  
  
  ---
  
  ### Answer by username_2 on 2022-11-10. Score: 0
  
  I share with you my experience with TensorFlow 2.0 (TF2.0). I don't know for other deep learning packages. In TF2.0, you must pass `x_train` and `y_train` together during training, where `x_train` is the image and `y_train` is the mask. (Similar holds for `x_test` and `y_test`, which must be passed together during evaluation.) This means that you must have a mask for both images with tumor and images without tumor. In your case, `y_train` would be a binary mask, since you have two pixel classes (tumor and no tumor). For images with no tumor, the mask would be all `0's` (what you call an "empty mask"). For images with tumor, the mask would be `0's` everywhere but `1's` where there is tumor (what you call the "patch"). Note that the masks are essentially images (or numpy arrays, if you prefer) of the same size as your medical images, except that the mask has binary pixel values according to tumor present (`1's`) or absent (`0's`).
  
  Thus, the answer to your question, *"Should I also feed my model empty masks along with patches (non empty mask)?"*, is `Yes`. In fact, in TF2.0, you absolutely must. (A separate post can be started if you have questions about model performance.)
  
  
  ---
  
  