The UK Independence Party is calling for David Cameron to apologise after he claimed its members were "fruitcakes, loonies and closet racists, mostly".
The Conservative leader made the remarks during a phone-in on the London radio station LBC.
He said he believed UKIP was "racist" because of some of the things he had heard its members say.
But UKIP Euro MP Nigel Farage said that unless he could prove his claims, Mr Cameron should apologise.
The comments came when Mr Cameron was asked about UKIP's plans to press for the names of secret Tory donors to be made public under the Freedom of Information Act.
He said the party was just trying to make mischief, adding: "UKIP is sort of a bunch of ... fruitcakes and loonies and closet racists, mostly."
After being told of the comments, Mr Farage said he believed it was the Tory leader's "inexperience as a politician that may account for why he stepped way over the line".
He said: "Let him tell us anything that we have said that could be construed as remotely racist, otherwise we demand an apology.
"I don't mind him calling us loonies - I don't mind him calling us fruitcakes. We are big enough and ugly enough and we have a sense of humour.
"But what you cannot do in the 21st Century is to lob about accusations of racism.
"That is an absolute disgrace. We are a non-racist, non-sectarian party and we always have been, and frankly we're a damn sight more cosmopolitan than Mr Cameron's Conservative Party."
'Cranks and gadflies'
He added there was "a tradition of the Conservatives being very rude about UKIP".
"After all, Michael Howard called us cranks and gadflies, and I'm wearing the gadfly tie today... I think it just shows that we've touched upon a nerve."
He said 2.7m people had voted for UKIP during the European elections 18 months ago.
"We are obviously worrying Mr Cameron - that can be the only explanation for this ill-considered and ill-tempered outburst."
In a later interview during a visit to Leeds, the Tory leader defended his remarks and said he did not think he had not spoken out of turn.
"I was making a general point that I think UKIP has a problem in that they are the sort of 'stop the world, I want to get off' party," he said.
"I don't think they've got anything to say to a modern country and I was being asked particularly about something they're doing at the moment but, no, I think [it was] a very sensible thing to say."
At the launch of the UKIP local elections campaign last week, Euro MP John Whittaker said the party hoped to make gains at the expense of the Conservatives.
The Tories, under new leader Mr Cameron, had moved away from Middle England and deserted core Conservative voters, he argued.
"UKIP stands for the restoration of civil pride, helping people to feel they are part of the local community," said Mr Whittaker.
'Cash for peerages'
The row comes as Prime Minister Tony Blair prepares to meet Mr Cameron to discuss rules on political funding.
At present anyone donating more than £5,000 has to be named - but people lending money do not have to be.
The fact people had lent millions of pounds to Labour and the Tories in 2005 emerged only after claims, which have been denied, of "cash for peerages".
Both parties are being probed by police and the Electoral Commission. Both have said they will name future lenders.
Labour has named 12 businessmen who loaned the party almost £14m in the run-up to the general election.
The Tories have named 13 backers who lent them £16m, and say they will show the Electoral Commission a list of names of other people who lent money but who have now been repaid and who do not want to be identified.
UKIP has threatened to use the Freedom of Information Act to try to unmask the secret Tory backers who lent the party £5m to bankroll its General Election campaign.