Thanks for your response.
In general, PyTorch does not automatically parallelize the forward pass in a model?
Computation time1477×145 8.09 KB
Can you please check the following results using the autograd profiler.
The computation time of forward pass of through a network of different batch sizes. If it would have been parallelized then for larger batch sizes it could reduce the computation time.
Also, I don’t understand why the CPU time avg and CUDA time avg are almost identical?
Thanks.