That code looks very tensorflow – you have to do initialization things at init time and runtime things in your training loop. So lines 2, 4, 5, and 6 should be in your training loop while 1 and 3 are in your initialization.