cocorocho:

tb.add_graph(network, images)


Ok i found the issue, it was this line bottlenecking the training.

tb.add_graph(network, images)

My training speed is good after getting rid of that line.