@dat_pham_thanh
DataParallel would replicate the model, scatter the input, and gather outputs in every iteration. So, if the input size is too small, the overhead of replicating the model might overshadow the benefits of parallelizing the computation. Besides what @pritamdamania87 suggested above, could you please also try with large batch size?