Hi,
The problem with for loops is two fold:

Running the python code for the for loops can be slow and running very small ops is usually less efficient than running big ones (this slows down both the forward and backward pass).
All the indexing ops are not free.
The number of operations they add to the computational graph is huge and so any operation with it becomes very expensive.