What is the coefficient of $x^3$ in expansion of $(x^2 - x + 2)^{10}$ 
What is the coefficient of $x^3$ in expansion of $(x^2 - x + 2)^{10}$.

I have tried:  
$$\frac{10!}{(3!\times7!)} \times (-x + 2)^7 \times (x^2)^3  $$
But got an incorrect answer $-15360$.

You just missed one term. Note that by using the binomial theorem we get 
$$(x^2-x+2)^{10}=(x(x-1)+2)^{10}=\sum_{k=0}^{10}\binom{10}{k}2^{10-k}x^k(x-1)^k$$
Hence, since $[x^3](x^k(x-1)^k)=0$ for $k\not\in \{2,3\}$, we have that
$$\begin{align}[x^3](x^2-x+2)^{10}&=[x^3]\sum_{k=2}^{3}\binom{10}{k}2^{10-k}x^k(x-1)^k\\&=\binom{10}{2}2^8(-2)+\binom{10}{3}2^7(-1)\\
&=-23040-15360=-38400.\end{align}$$
