I don’t fully understand your question.
Do you want to use the encoder output as the segmentation prediction and just train the encoder without the decoder part?
If so, what do you mean with your last sentence?



 Kevinkevin189:

So every decoder layer can generate a sepcific mask for this layer itself?


Is there a decoder for each encoder layer?
Note that when you use the “small” encoder output to train your model, you would need to resize your segmentation mask to the appropriate size which will either yield very small segmentation prediction for your model or probably “blurred” segmentations if you resize it to the original size.