Media playback is not supported on this device
The Barcelona forward curled home a free-kick after only 12 minutes, and Luan added a second after the break.
Brazil will now play Honduras in Wednesday's semi-finals after they won 1-0 against South Korea, thanks to Alberth Elis's winner.
Germany play Nigeria in the other last-four match.
The Africans beat Denmark 2-0 in their quarter-final, while Germany won 4-0 against Portugal.
Brazil's second pool win came after 0-0 draws with Iraq and South Africa, and a 4-0 victory over Denmark.
Neymar, the poster boy for the Games, finally got off the mark at the fourth attempt when he curled home from 25 yards out.
He then provoked a mass brawl between the two teams after a heavy challenge on Andres Roa, and five yellow cards were handed out.
Brazil made sure of a place in their third consecutive Olympic semi-final when Luan's effort from outside the area beat goalkeeper Cristian Bonilla.
Neymar scored his first goal of the Olympics as hosts Brazil beat Colombia 2-0 to reach the semi-finals of the men's football.