Question: I have somehow lost sight of the big picture in my study of quantum computing. I understand that we still don't know if quantum computers are more powerful than classical ones, in the sense of computational complexity. But I'm wondering: 

Since every quantum gate can be seen as a (unitary) matrix multiplication, and matrix multiplication can be done classically in polynomial time, where is the supposed advantage coming from? What prevents me from taking a polynomial size quantum circuit and implementing it as a polynomial number of matrix multiplications (that require polynomial time)? what is the operation that is classically unavailable?

Answer: 
Matrix multiplication is polynomial in the number of matrix elements. In quantum computing the number of matrix elements is basically the number of elements in the quantum state vector, which is *exponential* in the size of the input (the number of qubits).
To put slightly differently the Hilbert Space (H) for your quantum circuit is growing exponentially ($ \text{dim H} = 2^n$) where n is the number of qubits. There are a class of quantum circuits that can be implemented in polynomial time but they require that the gates in the quantum circuit are restricted to the Pauli group and/or the normalizer of the Pauli group(Gottesman Knill Theorem). This also restricts what kind of error models one employs in the quantum circuit. The basic reason is that there is an isomorphism one can define between the Pauli group and vector space over the finite field with elements. Multiplication of the Pauli group elements becomes a symplectic product of the vector space elements.