Solved by ptrblck in post #2 
              
              
                Your training seems to be instable and thus sensitive to the random seed. Sometimes using other parameter initializations might help stabilizing the training overall.