Solved by cocorocho in post #4 
              
              
                Ok i found the issue, it was this line bottlenecking the training. 

tb.add_graph(network, images) 

My training speed is good after getting rid of that line.