Question: I'm trying to evaluate $\frac{\partial (Wx)^T A (Wx)}{\partial W}$, where

* W is a $m \times n$ matrix
* x is a $n \times 1$ vector
* A is a $m \times m$ symmetric positive semi-definite matrix (e.g. a covariance matrix)

I did some calculations with simple concrete examples and it looks the answer can be $2AWxx^T$, but I am not sure how to prove it in general.

I am familiar with $\frac{\partial y^TAy}{\partial y} = 2Ay$ (e.g. $y=Wx$), but I am confused what $\frac{\partial Wx}{\partial W}$ would be like and how the chain rule would work with it.

Answer: 
For positives $x$, $y$ and $z$ we need to prove that
$$\sum\_{cyc}\frac{x+y+z}{x}-3\geq4\sum\_{cyc}\frac{x+y+z}{x+y}-12$$ or
$$\sum\_{cyc}\frac{1}{x}+\frac{9}{x+y+z}\geq4\sum\_{cyc}\frac {1}{x+y}$$
or
$$\sum\_{cyc}\frac{1}{x}+\frac{3}{\frac{x+y+z}{3}}\geq2\sum\_{cyc}\frac {1}{\frac{x+y}{2}},$$which is Popovicui for the convex function $f(x)=\frac{1}{x}.$

There are another ways for the proof of your inequality (I see at least 10 of them).

If you want, I can show something.
We dont really need Popoviciu's Inequality here,Basically use the following lemma.
If $x,y,z>0\in \mathbb{R}$ $$\dfrac{x}{y}+\dfrac{x}{z}\geq \dfrac{4x}{y+z}$$
Proof : Note that this is equivalent to $x\left(\dfrac{1}{y}+\dfrac{1}{z}-\dfrac{4}{y+z}\right)\Longleftrightarrow\dfrac{x(y-z)^2}{yz(y+z)}\geq 0$ which is clearly true.

Back to the original problem using our lemma we have $$\dfrac{y+z}{x}+\dfrac{z+x}{y}+\dfrac{x+y}{z}=\left(\dfrac{x}{y}+\dfrac{x}{z}\right)+\left(\dfrac{y}{z}+\dfrac{y}{x}\right)+\left(\dfrac{z}{x}+\dfrac{z}{y}\right)\geq \dfrac{4x}{y+z}+\dfrac{4y}{z+x}+\dfrac{4z}{x+y}$$