wasiahmad:

in case if I loop over the individual timestep, will it be inefficient in terms of time complexity


RNNs are inherently sequential. They are auto-regressive, meaning the input for timestep t contains the output for timestep (t-1), meaning you have to first calculae the output for timestap (t-1).
This is one reason why ‘attention is all you need’ is quite interesting.