$wave_i(x) = N(i/256.0, (1/10000)^2)$
i
classes = 256
abs(input - const) < eps
model = Sequential([
Dense(classes, input_dim=200),
Activation('sigmoid'),
Dense(classes * 2),
Activation('sigmoid'),
Dense(classes),
Activation('softmax'),
])
2.xxxx
0.07
0.09
0.1
def generator():
while 1:
data = [numpy.random.normal(i/255.0,1/10000.0,225).tolist() for i in range(0, classes)]
labels = to_categorical([i for i in range(0, classes)], classes)
yield (data,labels)
When you have a really simple relationship between input and output, such as the one you are exploring, then this may not play to the strengths of a neural network, which is flexible enough to fit any function but rarely does so perfectly. When you have a simple function, you may find you will spot the imperfections in the fit from a neural network and models other than neural networks will do a better job.
Some things you could potentially do to get a better fit (roughly in order of things I would try):
Try a different optimiser. You don't say which optimiser you are using, but the Keras library comes with a few choices.
Neural networks work better when training and predicting against input features that have been normalised. An effective choice is mean 0, standard deviation 1. In your case, if you pre-process each batch - when training and testing - like this:
data = (data - 0.5)/0.289, it may help.
Increase number of neurons in hidden layers, and/or change the activation function. Your ideal activation function here might even be shaped like a gaussian (so a single neuron could immediately tune to each class), but that isn't something you usually find in a NN library. Consider dropping the middle layer too, and just have e.g.
8*classes neurons in a single hidden layer before the softmax*.
Sample from your input examples in the generator instead of calculating one from each class each time. The generator is potentially too regular - I have seen the classic
xor example network get stuck in a similar way to your description when fed the same inputs repeatedly.
* The simpler network model would look like this:
model = Sequential([ Dense(classes * 8, input_dim=200), Activation('sigmoid'), Dense(classes), Activation('softmax'), ])