Yeh then you can sum both losses at the same time, as it will jointly optimize the problem.
Think that classifier’s gradients affect the segmentation too. If you see that classifier info degradates the segmentation then you can train them appart. But in that case it would be better to fully train the segmentator and then the classifier. I don’t find the point of iterate between both.