Question: Standard algorithms have been proposed to compute the partial derivative of eigenvalue and eigenvector w.r.t. the matrix, e.g. <http://www.win.tue.nl/casa/meetings/seminar/previous/_abstract051019_files/Presentation.pdf>.

However, as far as I know, no solution has derived for the partial derivative of the matrix w.r.t. its eigenvalue and eigenvector. Suppose the matrix $A\in \mathbb{R}^{n \times n}$ with eigenvalue $\lambda\in \mathbb{R}$ and eigenvector $X\in \mathbb{R}^{n}$, i.e. $ AX = \lambda X$,the problem is to compute $\frac{\partial A}{\partial \lambda}$ and $\frac{\partial A}{\partial X}$. Does anyone have some ideas? Thanks for your help!

Answer: 
Assuming the matrix has a full set of eigenvalues and eigenvectors, then you can write $A = X\Lambda X^{-1}$ where $X$ is the (columnwise) matrix of eigenvectors and $\Lambda$ is the diagonal matrix of eigenvalues. In this form, the derivative with respect to the eigenvalue is easy to see:
$$ \frac{\partial A}{\partial \lambda\_i} = X\_i [(X^{-1})^T\_i]^T$$
which is the rank-1 outer product between the left eigenvector and corresponding (suitably normalized) right eigenvector row.

With regards to the derivative with respect to the eigenvector, what do you expect the format of the result to be in, a rank-3 tensor?
$\def\L{\left}\def\R{\right}$Since no one has yet addressed it,
here is the second of the requested gradients.

Using the same assumptions as Victor, calculate the differential wrt $X$
$$\eqalign{
 A &= X\Lambda X^{-1} \\
dA &= dX\Lambda X^{-1} + X\Lambda dX^{-1} \\
 &= dX\Lambda X^{-1} - X\Lambda X^{-1}\,dX\,X^{-1} \\
 &= I\_n\,dX\Lambda X^{-1} - A\,dX\,X^{-1} \\
}$$
Vectorizing both sides of this expression yields the desired gradient
$$\eqalign{
da &= \Big((\Lambda X^{-1})^T\otimes I\_n - X^{-T}\otimes A\Big)\,dx \\
 &= \L(X^{-1}\otimes I\_n\R)^T\L(\Lambda\otimes I\_n-I\_n\otimes A\R)dx \\
\frac{\partial a}{\partial x}
 &= \L(X^{-1}\otimes I\_n\R)^T \L(\Lambda\otimes I\_n - I\_n\otimes A\R) \\
}$$