The men, a 27-year-old Algerian and a 22-year-old Nigerian, were arrested last month on suspicion of planning a terror attack.
A gun and a flag of the so-called Islamic State were found at their homes during police raids in the central city of GÃ¶ttingen.
But the men have never been charged.
The criminal proceedings were dropped because police never established whether the suspects had planned to carry out an attack.  Police say the two men are "dangerous".
Lower Saxony Interior Minister Boris Pistorius said the deportations would take place as soon as possible, and certainly before mid-April.
He said discussions were already under way with Algeria and Nigeria to facilitate this, and the two men would be subject to a "life-long re-entry ban", preventing their return to Germany.
It was not immediately clear if the two men would appeal against the decision.
Germany determines citizenship by the nationalities of one or both parents and also a person's place of birth.
The country has been on high alert since the Berlin Christmas market attack on 19 December, that left 12 people dead and dozens injured.

It is understood proposals for a £12m project at Fairy Glen would mean half an acre of wood is destroyed, Coed Cadw has claimed.
It has asked Snowdonia National Park Authority to reject RWE Innogy's plans.
The scheme would be capable of generating up to 13,000 megawatt hours of electricity each year, enough to supply around 2,700 households.
RWE Innogy revealed the plans three years ago and the authority has been carrying out a consultation.
It would involve getting water from the river above Penmachno bridge and diverting it through more than half a mile (1km) of pipeline around Fairy Glen, before returning it to the river near the River Lledr junction.
Rory Francis from Coed Cadw said: "When woodlands such as this, which are centuries old, are lost they cannot be replaced.
"We urge the Snowdonia National Park planning committee to say no to the  proposal and for the company involved to think again."
RWE Innogy said a consultation had taken place, and that the project had been designed to to minimise any environmental impact.
A spokesman said: "Let's not forget that one of the biggest threats to the environment and wildlife is climate change itself, and so it is important that we are able to deliver projects like this which make an important contribution towards tackling climate change."
Germany says it will deport two men born in the country but whose parents are foreign - the first such case in German history.
[NEXT_CONCEPT]
Woodland in Conwy is "under threat" if a hydro-electric scheme goes ahead, a charity has said.