In my network, it’s highly dynamic so that every data will run different path in the network. In this case, I can not do batch because the data in same batch do not have same work pipeline. But I want to speedup my training, which means, I need some kind of parallism to accelerate the training in the case that I can only train one data per forward procedure.
Any One have some suggestions?