At first I would try to overfit the single sample, so the batch size shouldn’t matter.
I would try to use Adam with an initial lr=1e-3 and xavier_uniform for the weights.
If your model never reaches a nearly perfect solution, you might want to add some capacity to it, i.e. making the layers bigger.