So here is what happens when I sync (with the residual network)
total time 52.23 loading time 11.10, cuda transfer time 0.10, forward time: 12.89, cost time 0.03, step time 28.09, average cost time with data 0.02, average cost time with item 0.01
whereas previously it was (with .item())
total time 51.80 loading time 10.91, cuda transfer time 0.10, forward time: 9.27, cost time 0.02, step time 2.63, average cost time 28.87
and with .data[0]
total time 41.51 loading time 10.99, cuda transfer time 18.51, forward time: 9.34, cost time 0.02, step time 2.65, average cost time 0.01


So now there is no difference between data[0] and item(0) and step time (time it takes for .backward + .step operations) increases. Now most of the time is spend on backward and forward calculations as expected.
I assume when one uses item instead of data, some of the sync happens in the next batch of data which seems to affect the gpu loading time (presumably because this is where sync happens when one uses data instead of item). This resolves the question mostly thanks. Still not sure why .data[0] decreases the overall time spent though. going to do somemore tests to see whether if it really decreases or my timers miss a sync step somewhere.