Your approach would thus mean to change the 2D use case to a 3D one, right?
Instead of classifying a single slice you would now consider a small volume of slices and predict all segmentation masks for these slices.
That sounds reasonable, as the depth dimension should also yield useful information besides the spatial dimensions.
Or would you still use the 2D model and just pass multiple slices to the model?