Multi-thread should work, and this is how DataParallel 21 is implemented (search for parallel_apply). But if my_module.calculate is composed of many CPU ops, you might see sequential execution due to GIL.