What is the coefficient of $x^5$ in $(1+x+x^2+x^3+x^4+x^5)^{17}$? I figured that $(1+x+x^2+x^3+x^4)^{17} = (1-x^6)^{17}*(1-x)^{-17}$ but don't know what else to do. 
I would really appreciate any help

Since
$$
\left( {1 + x + x^2  + x^3  + x^4 } \right)^{\,17}  = \left( {{{1 - x^5 } \over {1 - x}}} \right)^{\,17} 
$$
you might be interested to know that the general
case of what are somewhere called r-nomial coefficients corresponds to the O.G.F:
$$
F_b (x,r,m) = \sum\limits_{0\,\, \leqslant \,\,s\,\,\left( { \leqslant \,\,r\,m} \right)} {N_b (s,r,m)\;x^{\,s} }
  = \left( {1 + x +  \cdots  + x^{\,r} } \right)^m  = \left( {\frac{{1 - x^{\,r + 1} }}{{1 - x}}} \right)^m 
$$
where
$$
N_b (s,r,m)\quad \left| {\;0 \leqslant \text{integers  }s,m,r} \right.\quad  =
\sum\limits_{\left( {0\, \leqslant } \right)\,\,k\,\,\left( { \leqslant \,\frac{s}{r+1}\, \leqslant \,m} \right)} 
{\left( { - 1} \right)^k \binom{m}{k}
 \binom
 { s + m - 1 - k\left( {r + 1} \right) } 
 { s - k\left( {r + 1} \right)}\ }
$$
You can find a full explanation in this related post and in this paper
In your particular case, if it is $(1+ \cdots+x^4)^{17}$
$$
\eqalign{
  & N_b (5,4,17)\quad  = \sum\limits_{\left( {0\, \le } \right)\,\,k\,\,\left( { \le \,{s \over {r + 1}}\, \le \,m} \right)} {\left( { - 1} \right)^k \left( \matrix{
  17 \cr 
  k \cr}  \right)\left( \matrix{
  21 - 5k \cr 
  5 - 5k \cr}  \right)}  =   \cr 
  &  = \left( \matrix{
  17 \cr 
  0 \cr}  \right)\left( \matrix{
  21 \cr 
  5 \cr}  \right) - \left( \matrix{
  17 \cr 
  1 \cr}  \right)\left( \matrix{
  16 \cr 
  0 \cr}  \right) = \left( \matrix{
  21 \cr 
  5 \cr}  \right) - 17 = 20322 \cr} 
$$
or for $(1+ \cdots+x^5)^{17}$
$$
\eqalign{
  & N_b (5,5,17)\quad  = \sum\limits_{\left( {0\, \le } \right)\,\,k\,\,\left( { \le \,{s \over {r + 1}}\, \le \,m} \right)} {\left( { - 1} \right)^k \left( \matrix{
  17 \cr 
  k \cr}  \right)\left( \matrix{
  5 + 17 - 1 - 6k \cr 
  5 - 6k \cr}  \right)}  =   \cr 
  &  = \sum\limits_{\left( {0\, \le } \right)\,\,k\,\,\left( { \le \,{s \over {r + 1}}\, \le \,m} \right)} {\left( { - 1} \right)^k \left( \matrix{
  17 \cr 
  k \cr}  \right)\left( \matrix{
  21 - 6k \cr 
  5 - 6k \cr}  \right)}  = \left( \matrix{
  21 \cr 
  5 \cr}  \right) = 20349 \cr} 
$$
