Such unannounced inspections treated staff like "naughty children", says Brian Lightman, head of the Association of School and College Leaders.
On Monday, Ofsted said there would be 40 no-warning inspections this month.
These will test the feasibility of unannounced inspections, proposed after the Trojan Horse inquiries in Birmingham schools.
There were calls for inspections to take place without any advance warning after claims some schools in the city were being taken over by hard-line Muslim groups,
This followed concerns that some of the schools were able to conceal unacceptable behaviour when they knew Ofsted inspectors were about to call.
But head teachers' leader Mr Lightman said: "Moving to no notice for routine inspections is unnecessary and would be counter-productive. It stifles creativity and treats professionals like naughty children."
"Schools currently only receive half a day's notice. This is the absolute minimum time needed so that key staff and governors are available to meet with inspectors and staff can gather the facts and figures that inspectors need during their visit," said Mr Lightman.
He accepted that where there is an "urgent cause for concern", unannounced inspections could be appropriate, but it should not become the standard procedure.
This month Ofsted plans to test the idea of no-notice inspections, with a wave of two-day unannounced visits across England.
"I'm currently giving thought to whether Ofsted should move to more routine, no-notice inspections as part of our wider education inspection reforms, which we will be consulting on later this year," said Sir Michael.
"In the meantime, under our regional structure, inspectors are well-placed to use their local knowledge and contacts to identify where these sorts of problems may be taking hold so we can respond swiftly and report publicly on what we find."
Meanwhile Ofsted is investigating claims that a number of schools in academy trusts in Norfolk were given advance warning of inspections.
The education watchdog also revealed that inspectors have returned for monitoring checks on five of the Birmingham schools placed into special measures after the Trojan Horse investigations.
The National Union of Teachers says it opposes adopting a system of routine no-notice school inspections.
"For accountability to be meaningful, there needs to be proper professional and respectful dialogue," said the union's general secretary, Christine Blower.
"The government should look to and learn from the light touch accountability systems of high-performing countries such as Finland and New Zealand which are based on trusting schools and teachers to do the best by their students, rather than the issuing of threats or penalties."

Property prices are expected to grow by 5% a year over the next five years, according to the Royal Institution of Chartered Surveyors (Rics).
This prediction had edged down in recent months, Rics said.
Price rise expectations in London have dropped from an estimate of 9% a year in March to just under 5% now.
"What we are really seeing is some of the very strong upward momentum starting to come off the housing market, as a lack of supply, higher prices, more prudent lending measures and some of the talk from the Bank of England are creating a level of caution among sellers and buyers," said Simon Rubinsohn, chief economist at Rics.
The Rics survey is one of a number of different surveys that are published about the UK housing market.
Earlier this month, the Nationwide Building Society said that there had been signs that activity in the UK housing market was starting to "moderate".
However, prices have risen in most parts of the country in recent months, with London showing double-digit annual growth in property prices.
Head teachers have attacked the idea of Ofsted inspectors visiting schools in England without any warning.
[NEXT_CONCEPT]
Surveyors have reduced their expectations for house price growth in the UK, despite a continued shortage of homes for sale.