Mostly, It might be because of the length of computation graph due to for-loop.
In (1), it is a single operation that is done using Multi-cores in one go (in GPUs).
(2) has to loop back through the computation graph (data transfer to & from GPUs) and calculate gradient one by one.