Frequency response We consider the response $x(t)$ when the frequency of the input y(t) varies.
$$x''+bx'+kx=ky(t), y(t)=\cos \omega t$$
1) If $k=3.24$, what is the natural ($b=0$) frequency $\omega_n$?
I think we have $\omega_n=\frac{\sqrt{3.24}}{2\pi}$
2) If $k=3.24$ and $b=0.8$, what is the amplitude gain from the input $y(t)=\cos \omega_n t$?
I need to report the ratio of the amplitude of x to amplitude y.
Thank you

Homogeneous equation is
$$
x''+bx+kx=0
$$
Characteristic equation of which
$$
\lambda^2+b\lambda+k=0
$$
which has solutions
$$
\lambda_{1,2}=-\frac b2 \pm  \frac{\sqrt{b^2-4k}}2
$$
To have solution only in reals and keep order of ODE, you have two options.
First if $b^2-4k > 0$ you have two real solutions
$$
\lambda_{1,2} = \frac b2 \pm \tilde \lambda
$$
where $\tilde \lambda = \frac{\sqrt{b^2-4k}}2$ and general solutions $x_h = e^{-\frac b2t}\left (C_1 \cosh \tilde \lambda t + C_2 \sinh \tilde \lambda t \right)$. It's called Overdamped harmonic oscillator. This solution actually doesn't do any oscillations and exponentially dies out.
Second option, if you have $b^2-4k < 0$, so you have two complex solutions
$$
\lambda_{1,2} = -\frac b2 \pm i \tilde \lambda
$$
where $\tilde \lambda = \frac {\sqrt{4k-b^2}}2$ and general solution $x_h = e^{-\frac b2t} \left ( C_1 \cos \tilde \lambda t + C_2 \sin \tilde \lambda t\right)$. Concept of natural frequency or any frequency at all makes sense for this option. It can be defined as $\omega_0$ in
$$
\omega = \frac {\sqrt{4k-b^2}}2 = \sqrt{\frac {4k-b^2}4} = \sqrt{k-\frac {b^2}4} = \sqrt{\omega_0^2-\chi^2}
$$
so $\omega_0 = \sqrt k$ and $\chi = \frac b2$. It's also called undumped frequency. Latter serves as damping parameter. Even it's called frequency, it's angular frequency. As for the frequency (which is measured in Hertz), it's $\nu = \frac \omega {2\pi}$
When $k = 3.24$ and $b = 0$, natural or undumped angular frequency is $\sqrt {3.24} = 1.8$ and $\nu = \frac {1.8}{2\pi}$, so your answer is correct.
As for the inhomogeneous ODE, looking at the form of forcing term, you can guess the solution as $x_p = A\cos \omega t + B \sin \omega t$ so
$$
x_p' = \omega(-A \sin \omega t+B\cos \omega t) \\
x_p'' = -\omega^2 (A \cos \omega t + B \sin \omega t)
$$
After substitution to the ODE
$$
-\omega^2 (A\cos \omega t + B \sin \omega t) + b \omega (-A\sin \omega t+B\cos \omega t) + k(A\cos \omega t + B\sin \omega t) = k\cos \omega t \\
\left [(k-\omega^2)A+bB\omega \right ]\cos \omega t + \left [ (k-\omega^2)B-bA\omega\right ] \sin \omega t = k\cos \omega t
$$
After matching coefficient in front of sines and cosines
$$
(k-\omega^2)A+bB\omega = k \\
(k-\omega^2)B-bA\omega = 0
$$
so one can find $A$ and $B$
$$
A = \frac {k\left(\omega^2-k\right)}{(\omega^2-k)^2+b^2\omega^2} \\
B = \frac {kb \omega}{(\omega^2-k)^2+b^2\omega^2}
$$
and particular solution will be
$$
x_p = \frac {k\left(\omega^2-k\right)}{(\omega^2-k)^2+b^2\omega^2} \cos \omega t + \frac {kb \omega}{(\omega^2-k)^2+b^2\omega^2} \sin \omega t = \\
= \frac k{\sqrt{(\omega^2-k)^2+b^2\omega^2}} \left ( \frac {\omega^2-k}{\sqrt{(\omega^2-k)^2+b^2\omega^2}} \cos \omega t + \frac {b \omega}{\sqrt{(\omega^2-k)^2+b^2\omega^2}} \sin \omega t \right )
$$
since
$$
\left ( \frac {\omega^2-k}{\sqrt{(\omega^2-k)^2+b^2\omega^2}} \right)^2 + \left (\frac {b \omega}{\sqrt{(\omega^2-k)^2+b^2\omega^2}} \right )^2 = 1
$$
you can subsitute
$$
\cos \delta = \frac {\omega^2-k}{\sqrt{(\omega^2-k)^2+b^2\omega^2}} \\
\sin \delta = \frac {b \omega}{\sqrt{(\omega^2-k)^2+b^2\omega^2}}
$$
and particular solution will be 
$$
x_p = \frac k{\sqrt{(\omega^2-k)^2+b^2\omega^2}} \left ( \cos \delta \cos \omega t + \sin \delta \sin \omega t \right ) = \frac k{\sqrt{(\omega^2-k)^2+b^2\omega^2}} \cos \left ( \omega t - \delta\right)
$$
Amplitude ratio is
$$
AR = \frac k{\sqrt{(\omega^2-k)^2+b^2\omega^2}}
$$
Just substitute your values to get corresponding function of $\omega$.
