Question: How do we calculate
$$2{\vec d\cdot \vec m\over \vec m\cdot \vec m}\vec m-\vec d$$

given the data ${\vec d = (1, 0)}$ and ${\vec m = (1, 1)}$?

The answer should be $(0, 1)$, but I don't know why.

Answer: 
I suppose it's simpler if you compute 
$$ \lim\_{x->0}\frac{x^2(\cos{x})^2 - \sin{x^2}}{x^2\sin{x^2}} $$

Note that you can substitute $x^2$ for $\sin{x^2}$ as x tends to zero and $t$ for $x^2$, the above limit equals to 
$$\lim\_{t->0} \frac{t(\cos{\sqrt{t}})^2-\sin{t}}{t^2}$$

Taylor's theorem is indeed convenient, but you can also use L'Hopital's rule twice, which involves some computation. If we are to use L'Hopital's rule, substituting t for $x^2$ reduces loads of work, I suppose. 

The answer, by the way, is -1.
You can simplify your life by noting that 

$$\frac{1}{\sin x}-\frac{1}{x}=x\frac{x}{\sin x}\frac{x-\sin x}{x^3}\to 0$$

So $$\frac{(\cos x)^2}{\sin (x^2)}-\frac{(\cos x)^2}{x^2}\to 0$$

and the limit becomes 
$$\frac{(\cos x)^2}{x^2}-\frac{1}{x^2}$$
 which is much easier, even if you use the hospital rule.