Choose $a, b$ so that $\cos(x) - \frac{1+ax^2}{1+bx^2}$ would be as infinitely small as possible on ${x \to 0}$ using Taylor polynomial $$\cos(x) - \frac{1+ax^2}{1+bx^2} \text{ on } x \to 0$$
If $\displaystyle \cos(x) = 1 - \frac{x^2}{2} + \frac{x^4}{4!} - \cdots $
Then we should choose $a, b$ in a such way that it's Taylor series is close to this.
However, I'm not sure how to approach this. I tried to take several derivates of second term to see its value on $x_0 = 0$, but it becomes  complicated and I don't see general formula for $n$-th derivative at point zero to find $a$ and $b$.

The quick-and-dirty method:
\begin{align*}
f(x) = \frac{1 + a x^2}{1 + bx^2} &= (1 + a x^2) \left( 1 - b x^2 + b^2 x^4 - b^3 x^6 + \cdots \right) \\&= 1 - (b - a) x^2 + (b^2 - ab) x^4 - (b^3 - a b^2) x^6 + \cdots
\end{align*}
We want $b - a = \frac{1}{2}$ and $b (b-a) = \frac{1}{24}$, so that (at least) the first three terms in the Taylor series of $f(x)$ and $\cos x$ agree.  This implies that $b = \frac{1}{12}$ and $a = -\frac{5}{12}$;  with this choice, we have
$$
f(x) = \frac{1 - \frac{5}{12} x^2}{1 + \frac{1}{12} x^2} = 1 - \frac{x^2}{2} + \frac{x^4}{24} - \frac{x^6}{288} + \cdots
$$
which agrees with $\cos x$ up to the $\mathcal{O}(x^6)$ term.
