If you change the data size to 51320, the batch size will increase to 2566, as you are calculating it dynamically using:
training_batches = 20
batch_size_train = int(len(dataX)/training_batches)

Running the script with this batch size gives:
GPU: 0.44s/epoch
CPU: 3.27s/epoch