How to effectively calculate $\int_0^\pi \sin^3x \sin(nx)dx$, $n\in \mathbb{N}$ How to effectively calculate $$\int_0^\pi \sin^3x \sin(nx)dx$$
I think by the formula:  $$\sin 3x=3\sin x -4\sin^3x $$ So we get $$\sin^3x = \frac{3\sin x-\sin 3x}{4}$$ 
Plug in we obtain $$\frac{1}{4}\int_0^\pi (3\sin x-\sin 3x) \sin(nx)dx$$
Then how do I proceed?  

We proceed exactly as how you started, and then use the product-to-sum formula: $\sin(a) \sin(b) = \frac{1}{2} (\cos(a-b) - \cos(a+b))$. If $n \neq 1, 3$:
$$\begin{aligned}
I_n &= \displaystyle \int_0^{\pi} \sin^3(x) \sin(nx) \; \mathrm{d}x\\
&= \frac{1}{4} \displaystyle \int_0^{\pi} (3 \sin(x) - \sin(3x)) \sin(nx) \; \mathrm{d}x\\
&= \frac{1}{4} \displaystyle \int_0^{\pi} 3 \sin(nx) \sin(x) - \sin(nx) \sin(3x) \; \mathrm{d}x\\
&= \frac{1}{4} \displaystyle \int_0^{\pi} \frac{3}{2} \left ( \cos \left ((n-1) x \right ) - \cos \left ( (n+1) x \right ) \right ) - \frac{1}{2} \left ( \cos \left ( (n-3) x \right ) - \cos \left ( (n+3) x \right ) \right ) \; \mathrm{d}x\\
&= \frac{1}{8} \left [ \frac{3}{n-1} \sin \left ( (n-1) x \right ) - \frac{3}{n+1} \sin \left ( (n+1) x \right ) - \frac{1}{n-3} \sin \left ( (n-3) x \right ) + \frac{1}{n+3} \sin \left ( (n+3) x \right ) \right ]_0^{\pi}\\
&= 0
\end{aligned}$$
Thus, if $n \neq 1, 3$, then the integral evaluates to $0$. If $n=1$, we have:
$$\begin{aligned}
I_1 &= \frac{1}{4} \displaystyle \int_0^{\pi} \frac{3}{2} \left ( \cos \left ((n-1) x \right ) - \cos \left ( (n+1) x \right ) \right ) - \frac{1}{2} \left ( \cos \left ( (n-3) x \right ) - \cos \left ( (n+3) x \right ) \right ) \; \mathrm{d}x\\
&= \frac{1}{8} \displaystyle \int_0^{\pi} 3 - 3\cos(2x)- \cos(-2x) + \cos(4x) \; \mathrm{d}x\\
&= \frac{1}{8} \left [3x - 3 \sin(2x) + \frac{3}{4} \sin(4x) \right ]_0^{\pi}\\
&= \frac{3\pi}{8}
\end{aligned}$$
If $n=3$, we have:
$$\begin{aligned}
I_3 &= \frac{1}{4} \displaystyle \int_0^{\pi} \frac{3}{2} \left ( \cos \left ((n-1) x \right ) - \cos \left ( (n+1) x \right ) \right ) - \frac{1}{2} \left ( \cos \left ( (n-3) x \right ) - \cos \left ( (n+3) x \right ) \right ) \; \mathrm{d}x\\
&= \frac{1}{8} \displaystyle \int_0^{\pi} 3\cos(2x)-3\cos(4x)-1+\cos(6x) \; \mathrm{d}x\\
&= \frac{1}{8} \left [\frac{3}{2} \sin(2x) - \frac{3}{4}\sin(4x) - x + \frac{1}{6} \sin(6x) \right ]_0^{\pi}\\
&= -\frac{\pi}{8}
\end{aligned}$$
Thus, over all cases we have
$$I_n = \begin{cases}
0 \quad &\text{if } n \neq 1, 3\\
\frac{3\pi}{8} \quad &\text{if } n = 1\\
-\frac{\pi}{8} \quad &\text{if } n = 3
\end{cases}$$
