The thinking of using multiprocessing module to share objects between worker processes indeed works, thank you.
As the feature files I used has a huge total size, and cannot be identified simply with index, I used modified pylru.lrucache object, registered it in multiprocessing.managers.BaseManager and shared the cached content between processes. I’ll post the example later.