Generating functions and finding coefficient of $x^{3n}$ So I have this homework question I've been trying all day to answer, any help would be appreciated:
Determine using generating functions in how many ways can you choose $3n$ letters out of $\{a, b, c\}$ so that each letter is chosen at most $2n$ times?

As @user84413's answer tells you, you are after:
$\begin{align*}
[x^{3 n}] (1 + x + x^2 + \dotsb + x^{2 n})^3
  &= [x^{3 n}] \left( \frac{1 - x^{2 n + 1}}{1 - x} \right)^3 \\
  &= [x^{3 n}] (1 - 3 x^{2 n + 1} + 3 x^{4 n + 2} - x^{6 n + 3}) \cdot (1 - x)^{-3} \\
  &= ([x^{3 n}] - 3 [x^{n - 1}]) \sum_{k \ge 0} (-1)^k \binom{3}{k} x^k \\
  &= ([x^{3 n}] - 3 [x^{n - 1}]) \sum_{k \ge 0} \binom{k + 3 - 1}{3 - 1} x^k \\
  &= \binom{3 n + 2}{2} - 3 \binom{n + 1}{2} \\
  &= \frac{(3 n + 2) (3 n + 1)}{2} - 3 \frac{(n + 1) n}{2} \\
  &= 3 n^2 + 3 n + 1 \\
  &= (n + 1)^3 - n^3
\end{align*}$
