In Bucharest some protesters broke through police lines and headed towards government buildings.
Prime Minister Sorin Grindeanu says the move would ease overcrowding in prisons.
His critics say he is trying to release allies convicted of corruption.
They also want the proposals to be debated in parliament.
Mr Grindeanu's plan is for the measures to be implemented through emergency decrees that bypass parliament and do not require the signature of President Klaus Iohannis.
Those who would be due for release include several elected officials and magistrates convicted of corruption.
President Iohannis joined protesters in Bucharest on Sunday, saying they were rightly indignant.
"Several political officials who have judicial issues want to change the legislation and weaken the rule of law," he said.
"It is unacceptable to modify the law so that the cases of dozens, even hundreds, of politicians, are wiped out."
President Iohannis was elected in November 2014 and has vowed to stamp out entrenched corruption.
However, the chairman of the ruling Social Democrats, Liviu Dragnea, accused the president of "inciting social disorder and violence" and said the protest was the start of a coup.
A former minister, Mr Dragnea was himself given a suspended prison sentence in 2015 for using bribes and forged ballot papers to influence an impeachment vote in 2012 against then President Traian Basescu.

Authorities say the man orchestrated clashes with police during a large protest in Seoul on 14 November.
Han Sang-gyun, head of the Korean Confederation of Trade Unions has been in the temple ever since the protest.
Monks earlier warned police not to force their way inside, saying it would be a breach of their religious rights.
In the early evening, shortly after a deadline passed to give himself up, police sealed off the Jogye Temple in central Seoul and began the operation to arrest Mr Han.
Those at the protest in November had a diverse range of grievances, from President Park Geun-hye's business-friendly economic policies, to a decision to force the nation's schools to only use state-approved history textbooks.
The rally saw clashes between police using pepper spray and water cannon, and protesters, some of whom were armed with metal pipes and sharpened bamboo sticks.
The unforgiving police response prompted a smaller demonstration on Saturday - 14,000 people, rather than the 60,000 estimated at November's rally - to demand an apology from the president.
Ms Park had drawn the ire of labour unions and farmers for planning to make it easier to dismiss workers and to cap the salaries of senior employees. However, many of her labour market policies are more popular with the general public, according to polls.
The BBC's Stephen Evans in Seoul says there is an increasingly bitter division between South Korea's right of centre government and its left-wing critics, who accuse it of diminishing democracy.
Tens of thousands of people have taken to the streets of the Romanian capital Bucharest and other cities in protest at government plans to release around 3,000 prisoners.
[NEXT_CONCEPT]
Hundreds of police officers have begun an operation to arrest a trade union leader who has taken refuge in a Buddhist temple in Seoul.