More than 20 migrant workers - most thought to be Ethiopian Christians - were killed by the Libyan branch of IS.
It released videos on Sunday of some of the men being beheaded and others shot.
Ethiopia's prime minister warned the protesters about the dangers of illegal immigration and described the killings as "Satanic".
IS and other jihadist groups are active in many towns in Libya, which has been torn by civil conflict since last year - and has been unstable since long-time leader Muammar Gaddafi was toppled in 2011.
"This week's cruel act which was committed against our citizens in Libya not only gives a glimpse into terrorism, but also shows the Satanic acts and objectives of those who committed the act," Ethiopian Prime Minister Hailemariam Desalegn told the mass rally in Addis Ababa's Meskel Square.
He also urged unity in the fight against what he called "home-grown extremism" in Ethiopia, and said those who chose to an illegal route to migrate risked falling prey to human traffickers.
"It is clear to everyone that our fellow citizens all have the right to live and work in any part of the world. But the illegal migration that leads to unnecessary suffering and death carried out by illegal human traffickers must stop."
However, later the government-condoned protest broke into scuffles with some parts of the crowd throwing stones, chanting anti-government slogans and clashing with police.
Police fired rounds of tear gas at some towards the end of the demonstration, the AFP news agency reports.
The rally comes a month before Ethiopia holds parliamentary elections, the first since the death in 2012 of long-time leader Meles Zenawi.
Tens of thousands of Ethiopians have attended a rally in the capital, Addis Ababa, condemning the murders of Ethiopians by Islamic State militants.