Question: I am developing a neuroscience-inspired training algorithm for feed-forward neural networks. The natural benchmark for comparison is backpropagation. So I need to know to know what sort of classification problems backpropagation (and thus feed-forward NNs) is most suited for. Also, since I ultimately want to move beyond backprop, what non NN algorithms are the best alternatives to backprop, when it comes to the sort of problems that backprop is good at. 

Or, in other words, what problems should I benchmark a feed-forward network on, and what are the algorithms to beat for those problems?

Answer: 
We know $X-\mu$ has a $N(0,\sigma^2)$ distribution. So 

$$E( (X-\mu)^2 ) = {\rm var}(X-\mu) + [E(X-\mu)]^2 = \sigma^{2} + 0 = \sigma^2$$

therefore 

$$E\left(\frac{(X-\mu)^2}{\sigma^6}\right) = \frac{1}{\sigma^6} \cdot E( (X-\mu)^2 ) = \frac{\sigma^2}{\sigma^{6}} = \frac{1}{\sigma^4}$$

which implies

$$-E \left [\frac{1}{2\sigma^4}-\frac{(X-\mu)^2}{\sigma^6}\right] = -\frac{1}{2\sigma^4} + \frac{1}{\sigma^4} = \frac{1}{2\sigma^4}$$
If $\sigma$ is the standard deviation then it is positive (or at least non-negative). 

If $\sigma$ is a constant then $$-E\left[ \frac 2 \sigma \right] = -\frac 2 \sigma$$

In the expression for the [Fisher Information](http://en.wikipedia.org/wiki/Fisher_information) you can often find

$$
\mathcal{I}(\theta) = - \operatorname{E} \left[\left. \frac{\partial^2}{\partial\theta^2} \log f(X;\theta)\right|\theta \right]\,
$$

the sign is present because the expectation is negative. There is another expression $$\mathcal{I}(\theta)=\operatorname{E} \left[\left. \left(\frac{\partial}{\partial\theta} \log f(X;\theta)\right)^2\right|\theta \right]$$ withought the negative sign.