Solved by the-dharma-bum in post #2 
              
              
                Your example isn’t coherent with your explanation above. Given your explanation, seg and F size should be respectively (2, 2, 2) and (2, 3, 2, 2): 
seg = [[[0, 1],
        [2, 3]],
       [[1, 2],
        [1, 2]]]

mask = np.array(seg)

print(mask.shape)  # (2, 2, 2)

F = [[[[2, 3],
       [4, 3]],
…