The long story is

  

      github.com/tensorflow/tensorflow
  

  
    
  
	  
  

  
    
      Adding a utility to penalize majority class pixels in the Segmentation tutorial 55
    

    
      
        opened 
        
          
        
        Apr 23, 2021
      
      

        
          closed 
        
          
        
        May 4, 2021
      
        

      
        
          
          sayakpaul
        
      
    

    
        stat:awaiting tensorflower
        type:docs-bug
        type:feature
    
  


  
    Thank you for submitting a TensorFlow documentation issue. Per our GitHub
polic…y, we only address code/doc bugs, performance issues, feature requests, and
build/installation issues on GitHub.

The TensorFlow docs are open source! To get involved, read the documentation
contributor guide: https://www.tensorflow.org/community/contribute/docs

## URL(s) with the issue:

Please provide a link to the documentation entry, for example:
https://www.tensorflow.org/tutorials/images/segmentation

## Description of issue (what needs changing):

It's not really an issue, a suggestion rather. 

### Clear description

Semantic segmentation datasets can be highly imbalanced meaning that particular class pixels can be present more inside images than that of other classes. Since segmentation problems can be treated as per-pixel classification problems we can deal with the imbalance problem by weighing the loss function to account for this. It's a simple and elegant way to deal with this problem. Other solutions include always ensuring that a batch of samples (during training) always contain some proportion (which is prefixed) of positive classes. 

However, TensorFlow does not yet support the `class_weight` argument in `model.fit()` for targets that are 3D (for segmentation problems, we are essentially predicting a map of shape `[batch_size, height, width, nb_channels]`). One way to get around this problem is to use `sample_weight` instead. But then again, it's not very clear as to how to do that properly particularly with `tf.data` pipelines. 

Multiple folks have tried several hacks to get around this problem but it keeps coming back (see [here](https://github.com/keras-team/keras/issues/3653)). Therefore, I think the tutorial under question is a perfect opportunity to demonstrate the use case. 

Cc: @MarkDaoust
  

  

  
    
    
  

  



  

      github.com/keras-team/keras
  

  
    
  
	  
  

  
    
      Keras - how to use class_weight with 3D data 73
    

    
      
        opened 
        
          
        
        Sep 1, 2016
      
      


      
        
          
          bsafacicek
        
      
    

    
    
  


  
    Hi, 

I am using Keras to segment images to road and background pixels. As you c…an imagine percentage of road pixels are much lower than that of background pixels. Hence, I want to use class_weight= {0:0.05, 1:0.95} while fitting the model so that cnn won't predict every pixel as background. But, when I do this I got the following error:

  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 597, in fit
    sample_weight=sample_weight)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1035, in fit
    batch_size=batch_size)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 973, in _standardize_user_data
    in zip(y, sample_weights, class_weights, self.sample_weight_modes)]
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 387, in standardize_weights
    raise Exception('class_weight not supported for '
Exception: class_weight not supported for 3+ dimensional targets.

My  training labels are in this form: (number_of_training_samples=10000, number_of_pixels_in_patch=16384, number_of_classes=2). How can I weight the classes in Keras?

Thanks in advance.
  

  

  
    
    
  

  


/cc @Sayak_Paul @markdaoust