I could reproduce the slower inference time of the model within the segmentation app in the benchmark app by waiting for some milliseconds. The pre- and post-processing takes about 115ms, so by sleeping for 115ms in between inference calls in the benchmark app, the performance slows down from 27ms/inference to the 45ms/inference I also observe in the segmentation app. This seems to happen gradually until it maxes out at about 45ms:

no sleep -> 27ms/inf
sleep for 50ms -> 32ms/inf
sleep for 75ms -> 35ms/inf
sleep for 100ms -> 39ms/inf
sleep for 115ms -> 43ms/inf
sleep for 500ms -> 44ms/inf

Is my observation correct and is there any way to prevent the model from slowing down as a result of time spent between forward calls? I would like it to perform at 27ms/inference in the segmentation app as well.