PyTorch will add the CUDA kernels to the queue and will launch them in parallel if enough resources are available. This post describes this behavior in more detail with some examples.
Since you are using linear layers, the matmul operation would most likely use the entire compute resources and thus no parallel execution might be possible.