I tried to reproduce the results by using the code provided in the tutorial on Single Machine Model Parallel Best Practices 6, however the results were a bit different.

The model with pipelining is expected to perform better than than the rest two cases but it doesn’t. What can be the possible reasons for this?