I’m unsure which dimension the sequence is in. Based on your description it seems as if you are dealing with a sequence of images so are you using a 3D-Unet with the temporal dimension (sequence) as the “depth”?