That’s an interesting use case. However, would you like to use 3D modules for it (e.g. nn.Conv3d) or rather a windowed approach?



 aksg87:

I was also wondering if there was a good way to check consistency across the three image predictions as an input to the model. (I.e. the predictions should have a similar distribution and number of positive pixels should also be similar – the anatomy cannot shift rapidly across a single image).


Really just a wild idea, but for a windowed approach, you could use the predicted classes of the “middle” slice as an additional cross entropy loss for the neighboring slices.
I have no idea, if this would work at all, but might be worth a try.