Good day vertias,
without knowing details about the exact memory footprint, architecture and payload of inferences, it is not easy to give detailed help, but:

Have you taken a look at the mobile Interpreter 2? This may reduce the memory footprint by up to 75%.
Consider spending more time on producing an end-to-end architecture, since this will save all your problems in one go. A good segmentation architecture should in general be capable of segmenting streets AND cars.
Reduce the architecture of your model by applying Knowledge Distillation 1.

Cheers.