Solving $\int_0^{\infty} \sin(ax^2)\sin(b/x^2)\,\mathrm dx$ The integral is:
$$
\int_{0}^{\infty}\sin\left(ax^{2}\right)\sin\left(\frac b {x^{2}}\right)\,\mathrm{d}x
$$
I already know the answer but I cant prove it! ( You can get the answer using Maple ). I think it should be simplified to a couple of Gaussian integrals ( $\operatorname{erf}$ functions ) but it's not so obvious.

$$\int_0^{\infty} \sin ax^2\sin bx^{-2}\,dx=\frac{1}{2}\int_0^{\infty} \cos \left(ax^2-\frac{b}{x^2}\right)\,dx-\frac{1}{2}\int_0^{\infty} \cos \left(ax^2+\frac{b}{x^2}\right)\,dx$$
In general:
$$\begin{aligned} \int_{0}^{\infty}f \left(p x-\frac{q}{x}\right)^2\,dx &= \sqrt{\frac{q}{p}}\int_{0}^{\infty  }f \left(\sqrt{pq} x- \frac{\sqrt{pq}}{x}\right)^2\,dx \quad(\textstyle x\to  \sqrt{\frac{q}{p}}x )\\ &=\sqrt{\frac{q}{p}}\left[ \int_{0}^{1} f \left(\sqrt{pq} x-\frac{\sqrt{pq}}{x} \right)^2\,dx+ \int_{1}^{\infty} f \left(\sqrt{pq}x-\frac{\sqrt{pq}}{x}\right)^2\,dx \right]\\&= \sqrt{\frac{q}{p}}\left[\int_{1}^{\infty} \left(1+\frac{1}{x^2}\right)f \left(\sqrt{pq} x-\frac{\sqrt{pq}}{x} \right)^2\,dx \right]\quad( x\to x^{-1})\\&=\frac{1}{p} \int_{0}^{\infty} f (t^2 )\,dt \quad \big(t=\sqrt{pq} x-\sqrt{pq}x^{-1}\big) \end{aligned}$$
Observe that:
$$\begin{aligned}\int_0^{\infty} \cos \left(ax^2+\frac{b}{x^2}\right)\,dx&=\cos 2\sqrt{ab}\int_{0}^{\infty} \cos \left(\sqrt{a}x-\frac{\sqrt{b}}{x}\right)^2\,dx\\&\;-\sin 2\sqrt{ab}\int_{0}^{\infty} \sin \left(\sqrt{a}x-\frac{\sqrt{b}}{x}\right)^2\,dx\end{aligned}$$
Now since $\displaystyle \int_0^{\infty}\cos x^2\,dx=\int_0^{\infty}\sin x^2\,dx=\sqrt{\frac{\pi}{8}}$ using our previous result we have:
$$\int_0^{\infty} \cos \left(ax^2+\frac{b}{x^2}\right)\,dx=\sqrt{\frac{\pi}{8a}}\left(\cos 2\sqrt{ab}-\sin 2\sqrt{ab}\right)$$
Similarly, writing:
$$\begin{aligned}\int_0^{\infty} \cos \left(ax^2-\frac{b}{x^2}\right)\,dx&=\cosh 2\sqrt{ab}\int_{0}^{\infty} \cos \left(\sqrt{a}x-i\frac{\sqrt{b}}{x}\right)^2\,dx\\&\;-\sinh 2\sqrt{ab}\int_{0}^{\infty} \sin \left(\sqrt{a}x-i\frac{\sqrt{b}}{x}\right)^2\,dx\end{aligned}$$
We get:
$$\int_0^{\infty} \cos \left(ax^2-\frac{b}{x^2}\right)\,dx=\sqrt{\frac{\pi}{8a}}\left(\cosh 2\sqrt{ab}-\sinh 2\sqrt{ab}\right)$$
Putting everything together:
$$\int_0^{\infty} \sin ax^2\sin bx^{-2}\,dx=\frac{1}{2}\sqrt{\frac{\pi}{8a}}\Bigg\{\big(\cosh 2\sqrt{ab}-\sinh 2\sqrt{ab}\big)-\big(\cos 2\sqrt{ab}-\sin 2\sqrt{ab}\big)\Bigg\}$$
