The grenade was found during the night near buildings housing 170 people in the town of Villingen-Schwenningen. Its pin had been pulled out but the explosives failed to detonate.
Justice Minister Heiko Maas said it was a new level of "hate and violence".
There were 1,005 attacks on refugee homes in Germany last year - five times more than in 2014.
Some of the migrants at the Villingen-Schwenningen hostel were evacuated while bomb disposal experts carried out a controlled explosion.
Police spokesman Thomas Kalmbach said it was "just luck" no-one was hurt.
Officers said the grenade still contained its explosives but it was not clear whether it still had a detonator.
Mr Maas said he was summoning his regional state counterparts to crack down on hate crime.
"Grenades are already flying towards refugee homes. We can't wait until there is someone dead," he said.
News of the grenade incident came as new figures showed five times more attacks were carried out on migrant hostels in Germany last year than in 2014.
The total for 2015 was 1,005, compared with 199 in 2014, the police report said. Far-right activists are suspected in 90% of the cases.
Last year a record 1.1 million people sought asylum in Germany - many from war-torn Syria. Many local authorities have struggled to house them.
Germany is expanding its list of safe countries of return, hoping to curb the influx.
The governing coalition plans to declare Morocco, Algeria and Tunisia safe countries of origin, making it easier to send migrants back, said Vice-Chancellor Sigmar Gabriel.
Last year Germany did the same for several Balkan nations - including Albania, Bosnia-Herzegovina and Kosovo - to cut the large numbers of migrants claiming asylum. Very few of their applications are granted.
The police say the biggest rise in attacks on migrant hostels last year was registered in the mainly industrial state of North Rhine-Westphalia.
State Interior Minister Ralf Jaeger said "investigators have noticed a marked increase in aggressive language" towards migrants on the internet.
Most of the thousands of migrants arriving daily on Greek islands hope to get asylum in Germany.
Migrant crisis: Who does the EU send back?
A note on terminology: The BBC uses the term migrant to refer to all people on the move who have yet to complete the legal process of claiming asylum. This group includes people fleeing war-torn countries such as Syria, who are likely to be granted refugee status, as well as people who are seeking jobs and better lives, who governments are likely to rule are economic migrants.
Unidentified attackers threw a live hand grenade at a migrant hostel in south-western Germany, officials say.