Body: <p>Currently, I'm trying to optimize a training process of a neural net to improve final results. The problem I'm dealing with is multiclass segmentation on microscopic data.</p>
  <p>The paradox is that the best (and not sufficient) result is giving the simplest U-Net architecture on the original dataset. If I try a deeper or more complex model (e.g. r2unet), the final segmentation is significantly worse. If I try on the fly augmentation - worse as well. Changing a complex model into a more shallow one didn't help either (just tried out the other way than making it more complex).</p>
  <p>Now, I'm trying to make a custom loss function work to improve the segmentation.</p>
  <p>Any ideas what might be a root cause? Or any other ideas that could improve the result?</p>
  <p>To get more specific, here's an example of the data. The initial 4000x4000 images are cut to 512x512, which results in a little over 3700 images. Most of them don't include the classes and is just background, that's why I'm trying to make another loss functions work, as well as weighted classes.</p>
  <p><a href="https://i.sstatic.net/hEN62.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/hEN62.png" alt="example1" /></a></p>
  <p><a href="https://i.sstatic.net/8jReT.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/8jReT.png" alt="example2" /></a></p>
  <p><a href="https://i.sstatic.net/0wVBH.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/0wVBH.png" alt="example3" /></a></p>
  <p>So far, I'm using categorical cross-entropy as a loss function, however, dice, Jaccard, the focal losses seem could be more suitable and once I'll finish my computations I'll try to make these work again, so far my tries weren't really compatible with Keras, at least it seemed.</p>
  <p>The size of U-Net</p>
  <ul>
  <li>depth = 5</li>
  <li>first conv layer has 64 filters, goes up to 1024.</li>
  </ul>
  <p>R2U-Net</p>
  <ul>
  <li>depth 5</li>
  <li>first layer 64filter (tried also 32)</li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-08T14:54:24.980Z
Comments.Text: Would be helpful to know more info about your training regime. How many images (an example would be nice as well)? Dice, focal loss, cross entropy, something else? U-Net initialize how?
Comments.UserDisplayName: Edoardo Guerriero
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-14T23:15:34.557Z
Comments.Text: You're using only 3700 training instances? That seems too little. Also, how many test instances do you have?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-15T13:22:17.087Z
Comments.Text: @nbro - at this very moment I have two versions of data I'm testing to use
  1) these 3700 rotated each by 90,180,270 degrees = almost 15k
  2) on the fly augmentation (horizontal, vertical flipping, cropping, sliiight blurring, rotating from -180 to 180 degrees, all with a certain probability of happening). The paradox is that the simple, 15k datapoint set makes significantly better results.
  
  the 3700/15k is a total number of images that I further split to train, test, validation parts by 75/15/10
Comments.UserDisplayName: Nuwanda
Tags: <tensorflow><keras><image-segmentation><u-net><categorical-crossentropy>
Title: Why is the simplest U-Net architecture giving the best (but not good enough) results on a multi-class segmentation on microscopic data?
UserDisplayName: Nuwanda
ThreadText: # Why is the simplest U-Net architecture giving the best (but not good enough) results on a multi-class segmentation on microscopic data?
  
  (Asked by: username_0 on 2021-12-08)
  
  Currently, I'm trying to optimize a training process of a neural net to improve final results. The problem I'm dealing with is multiclass segmentation on microscopic data.
  
  The paradox is that the best (and not sufficient) result is giving the simplest U-Net architecture on the original dataset. If I try a deeper or more complex model (e.g. r2unet), the final segmentation is significantly worse. If I try on the fly augmentation - worse as well. Changing a complex model into a more shallow one didn't help either (just tried out the other way than making it more complex).
  
  Now, I'm trying to make a custom loss function work to improve the segmentation.
  
  Any ideas what might be a root cause? Or any other ideas that could improve the result?
  
  To get more specific, here's an example of the data. The initial 4000x4000 images are cut to 512x512, which results in a little over 3700 images. Most of them don't include the classes and is just background, that's why I'm trying to make another loss functions work, as well as weighted classes.
  
  [![example1](https://i.sstatic.net/hEN62.png)](https://i.sstatic.net/hEN62.png)
  
  [![example2](https://i.sstatic.net/8jReT.png)](https://i.sstatic.net/8jReT.png)
  
  [![example3](https://i.sstatic.net/0wVBH.png)](https://i.sstatic.net/0wVBH.png)
  
  So far, I'm using categorical cross-entropy as a loss function, however, dice, Jaccard, the focal losses seem could be more suitable and once I'll finish my computations I'll try to make these work again, so far my tries weren't really compatible with Keras, at least it seemed.
  
  The size of U-Net
  
  * depth = 5
  * first conv layer has 64 filters, goes up to 1024.
  
  R2U-Net
  
  * depth 5
  * first layer 64filter (tried also 32)
  
  **Tags:** tensorflow, keras, image-segmentation, u-net, categorical-crossentropy
  > **Comments:**
  > **username_1**: Would be helpful to know more info about your training regime. How many images (an example would be nice as well)? Dice, focal loss, cross entropy, something else? U-Net initialize how?  
  > **username_2**: You're using only 3700 training instances? That seems too little. Also, how many test instances do you have?  
  > **username_0**: @nbro - at this very moment I have two versions of data I'm testing to use 1) these 3700 rotated each by 90,180,270 degrees = almost 15k 2) on the fly augmentation (horizontal, vertical flipping, cropping, sliiight blurring, rotating from -180 to 180 degrees, all with a certain probability of happening). The paradox is that the simple, 15k datapoint set makes significantly better results.  the 3700/15k is a total number of images that I further split to train, test, validation parts by 75/15/10  
  
  ---
  
  ## 0 answers
  
  