The complete code is here https://gist.github.com/zeakey/9d1c313329a7ea32ea12ae0f3a8db09f 24.
Actually I do plot the averaged loss/precision.
But it still makes me confused why the first batch always performs the best.