Opposition councillors accused the ruling Labour group of "arrogance" and not listening to the public's concerns.
The plans had also prompted protests outside the council's offices on Wednesday.
Council leader Peter Marland said he was "disappointed" but would work with other parties to agree a new budget.
The authority's provisional 2015-16 budget had included £1.5m cuts to bus services, and reduced funding for the Citizens' Advice Bureau.
Homeless shelters faced possible closure under the plans, and the removal of other funding could have forced some scout groups to shut down.
Liberal Democrat leader, Douglas McCall, accused council leaders of refusing to listen to public concerns or engage with opposition parties.
He described their approach to a public consultation as "arrogant".
However, Mr Marland said a record consultation response showed the authority was listening.
"What we couldn't do is persuade enough opposition politicians that what we were doing was ultimately right," he said.
Former Conservative group leader Andrew Geary said his party had predicted savings for five years and put forward a "clear strategy" for delivering them when it was in control.
"You simply can't amend something that is not built on a solid foundation and this budget is built on the sand and it will collapse and it won't deliver."
The authority is set to meet again next week to discuss the revised proposals.
Analysis
Paul Scoins, Political reporter, BBC Three Counties
After the political grandstanding comes the negotiation.
Concessions will be made but the key sticking point of provision for homeless people will remain a subject for debate.
The council will then need to agree that budget and, if they fail, the government will step in and act, most likely freezing the budget in its current state.
More than £90m needs to be saved over the next few years and Labour says difficult decisions are inevitable.
Sources within the leadership group say they would be surprised if an agreement cannot be reached, suggesting the "nuclear option" of handing the decision to communities secretary Eric Pickles would be something all parties would find difficult to swallow.

A pair of early strikes set them on their way. Defender Moussa Diarra netted his first goal in Barrow colours, not long after Danny Livesey had hit the bar.
Within three minutes, the Bluebirds' new goalscoring machine, Byron Harrison, grabbed his 12th goal since signing from Chesterfield in the summer.
Harrison nearly made it 3-0 when he hit the woodwork in the second half. Instead, Akwasi Asante went down the other end and was more clinical with his finish.
Jamey Osborne then hit the bar as Solihull rallied strongly, but the push was ultimately fruitless.
Report supplied by the Press Association.
Match ends, Barrow 2, Solihull Moors 1.
Second Half ends, Barrow 2, Solihull Moors 1.
Substitution, Barrow. Lindon Meikle replaces Jordan Williams.
Jamey Osborne (Solihull Moors) is shown the yellow card for a bad foul.
Substitution, Barrow. Andy Haworth replaces Alex-Ray Harvey.
Substitution, Solihull Moors. Harry White replaces Eddie Jones.
Substitution, Barrow. Akil Wright replaces Richard Bennett.
Eddie Jones (Solihull Moors) is shown the yellow card for a bad foul.
Substitution, Solihull Moors. Omari Sterling-James replaces Darryl Knights.
Nick Wilmer-Anderton (Barrow) is shown the yellow card for a bad foul.
Goal!  Barrow 2, Solihull Moors 1. Akwasi Asante (Solihull Moors).
Second Half begins Barrow 2, Solihull Moors 0.
First Half ends, Barrow 2, Solihull Moors 0.
Jack Byrne (Solihull Moors) is shown the yellow card for a bad foul.
Jordan Gough (Solihull Moors) is shown the yellow card for a bad foul.
Goal!  Barrow 2, Solihull Moors 0. Byron Harrison (Barrow).
Goal!  Barrow 1, Solihull Moors 0. Moussa Diarra (Barrow).
First Half begins.
Lineups are announced and players are warming up.
Proposals for £22m council budget cuts in Milton Keynes have been rejected over fears they would adversely affect vulnerable people.
[NEXT_CONCEPT]
Barrow took their incredible run of form to 18 matches unbeaten with a 2-1 home win over Solihull.