Your concern about the GIL might be valid (I haven’t verified it with your code though).
Since your method seems to be similar to HOGWILD, you could have a look at the HOGWILD MNIST example 46, which uses multiprocessing and would thus avoid the GIL.