The model is looking like the following:

model.png1703×1015 270 KB

Im preprocessing the optical flow with FlowGenerationNetwork (PWCNet) and the segmentation features (with Deeplab) and inputting it in the final network, for calculating the Lovasz loss.
…