Radical prison reforms to "shut the revolving door of crime and reoffending" have been outlined by Justice Secretary Ken Clarke.
Criticising the growth of the prison population in England and Wales, he said there must be other penalties.
The new measures will involve paying private firms and voluntary groups according to how many prisoners they rehabilitate.
He said locking people up for the sake of it was a waste of public funds.
David Cameron defended the plans during prime minister's questions and said it was now time for "radical reform" of the prison service.
But former Conservative Home Secretary Michael Howard has told Radio 4's PM programme he was "not convinced" by Mr Clarke's suggestion that fewer criminals should be sent to prison.
In his first major speech since taking office, Mr Clarke said prison had too often proved "a costly and ineffectual approach that fails to turn criminals into law-abiding citizens".
Speaking in London he said: "More than half of the crime in this country is committed by people who have been through the system. We must now take action and shut off this revolving door of crime and reoffending.
"We need a more constructive approach that tackles this head-on. An intelligent and transparent approach to sentencing that targets the causes of reoffending, so making our communities safer and better places to live. We describe it as a rehabilitation revolution."
Mr Howard, who was home secretary from 1993 to 1997, said he stood by his "prisons work" speech given to the 1993 Conservative Party conference.
Stressing that he was expressing his personal opinion, he said: "I've read [Mr Clarke's] speech.
"I'm not convinced by it.
"I think in order to protect the public, serious and persistent criminals need to be put in prison.
"I think that is what conscientious judges and magistrates are doing at present and I think that is the best way of protecting the public."
The prime minister said the prison service was "not working properly" at the moment.
He said: "When you've got these reoffending rates, when you've got the cost of each prison place, when you've got the appalling problem of drugs in prisons, we have got to reform."
He added: "If ever there was part of our public services that needed radical reform to make sure prison does work, then now that's it."
In May the prison population in England and Wales reached a record high of 85,201.
That is almost twice the figure in 1992, when Mr Clarke was last in charge of prisons as home secretary.
It was an increase he said he would have dismissed as "impossible and ridiculous" if someone had predicted it at the time.
Earlier, speaking on BBC Radio 4's Today programme, Mr Clarke said a sentencing review based on punishment and protecting the public was needed.
Criticising the increase of the prison population under Labour, he added: "[Former Home Secretaries] David Blunkett and John Reid had a chequebook in one hand and a copy of the Daily Mail in the other."
The government plans to use the voluntary and private sectors to rehouse, rehabilitate and find work for released prisoners, with bodies only paid if their efforts lead to a significant fall in reoffending.
These reforms will hopefully pay for themselves by reducing the costs elsewhere within the criminal justice system, Mr Clarke added.
But shadow home office minister David Hanson said Mr Clarke needed to back his ideas with a better prisons budget.
"Where Ken Clarke has a point is that there are people who go through the system who revolve through that door who have long-term drug, alcohol, employment problems - who need to be reformed to get back into society in a positive way," he said.
"But where I disagree with Mr Clarke is that it takes resources. And he is now planning to cut the resources to the justice department and to probation by 25% over the next four years."
And shadow justice minister Jack Straw said the increased number of offenders sentenced to prison had been a key factor in reducing crime.
"Does anyone seriously believe that crime would have come down and stayed down without these extra prison places?" he wrote in the Daily Mail.
The general secretary of the Prison Governors Association, Paddy Scriven, said jails had some "stunning successes" with inmates who spent more than six months behind bars.
"If we're left to concentrate on that and the shorter sentenced prisoners become the responsibility of a properly-funded probation service, then ...we're cutting costs and looking after the public in the way that we should be," he said.
'Poor value for money'
In a statement, the Probation Chiefs Association said it supported Mr Clarke's plans and said the next step for the service should be a "flexible strategic framework free of overly bureaucratic control and interference".
Juliet Lyon, of the Prison Reform Trust, said the system needed change as it was not working.
"You only have to look at the reconviction rates for those short sentence people - over 60% reconvicted within a year of release - to realise that we've got very poor value for money," she said.
Reoffending rose by 8% for adults between 2006 and 2008, and nearly half of offenders sent to prison are reconvicted within a year.
Meanwhile, MSPs have passed proposals to vastly reduce the number of prison sentences of three months or less at the Scottish Parliament, despite opposition from Conservatives.
The measure, which will introduce a presumption against the shorter sentences by sheriffs, was voted through in the Criminal Justice and Licensing Bill by MSPs at Holyrood.