From the moment he became leader of the party in September 2001 Iain Duncan Smith could never claim the backing of the majority of Tory MPs.
Duncan Smith: Opposed from start
He upset the odds by fighting off heavyweights Ken Clarke and Michael Portillo to claim the crown after the Conservatives' second successive landslide general election defeat.
But the manner of his election could be said to have sowed the seeds for much of the trials and tribulation he has suffered in Westminster.
Among Tory MPs he came second with 54 votes in the three-way run off to choose the two candidates to go forward to a vote of party members.
The support from these (mostly Eurosceptic) MPs might have been enough to take him through to the final vote by grass roots party members - but they made up less than a third of Tory MPs.
A further difficulty was a number of Tory MPs from the Major era believed his own political career made it difficult to demand loyalty from his party.
DUNCAN SMITH'S CV
Born April 1954
Joined Tory Party in 1981
Contested Bradford West 1987
Elected as MP for Chingford 1992
Social security shadow 1997-99
Defence shadow 1999-2001
Party leader 2001-
This was because he had often been at the centre of the parliamentary Euro-troubles that seriously destabilised John Major's government.
He was one of those Tories helping to orchestrate vote after vote against his own side on the Maastricht Bill in 1992 and 1993.
These factors all meant that, once there were signs the public were not warming to him, there were plenty of people within his own ranks not unhappy to see him in difficulty.
With that lack of a ringing endorsement it would need a stunning start to win over the doubters.
The former military man admitted as much himself, when he said his first 100 days would be crucial.
As it turned out, his election as leader came a day or two after the 11 September attacks on the US - and the new leader struggled to get his name, face or ideas noticed amongst the birth of the war on terror.
From this point onwards, while not noticeably doing anything wrong, the mutterings began among commentators and Tory MPs that he was not exactly leading a revival of fortunes.
A succession of ill-fated internal party appointments also added to the ranks of potential opponents.
Elected as a Eurosceptic right-winger many of his speeches and early appointments appeared to have moved the party towards the centre, with efforts taken to project a more caring image.
However, despite a series of policy announcements, the move failed to win over the bulk of the modernisers, while simultaneously chipping away at his support among the Tory right wing.
All these factors together meant that
Mr Duncan Smith was never able to escape the background chatter from his internal opponents.
There were a number of doomed attempts to end the plotting talk which continued despite opinion polls fairly consistently showing an upward trend for the Tories during his time as leader.
At one point, the reports forced him to call a press conference at Conservative Central office and issue a direct "unite or die" warning to his party.
At the 2002 Tory conference he warned his doubters not to underestimate "the quiet man", telling his detractors that he would not shirk in his mission to transform his party.
The first blatant challenge to IDS came on the night of the 2003 local elections when frontbench spokesman, Crispin Blunt, quit his post to voice his concern.
The results subsequently saw the Tories make significant gains on Labour and the attack fell away.
The issue went quiet during a summer when the main leadership speculation was about Mr Blair, suffering his own internal party difficulties post Iraq and post-Hutton.
However the backstage talk of plotting returned with a vengeance and overshadowed Mr Duncan Smith's launch of a raft of well received new policies at this year's Tory conference.
He told the Tory faithful in Blackpool that the "quiet man" was "turning up the volume" and warned the malcontents to "get on board" or get out of the way.
He accused Tony Blair of lying about the Dr David Kelly tragedy and joked about Liberal Democrat leader Charles Kennedy's drinking - personal attacks which raised eyebrows.
The speech received a host of standing ovations in the conference hall but outside there was criticism of the personal attacks and of his speaking style.
There were reports that he had been given public speaking training, which were seized on by opponents.
In the House of Commons his manner and performances also came under close scrutiny, with many comparing him unfavourably with his predecessor William Hague.
Most recently Mr Duncan Smith faced allegations about how he employed his wife in his private office.
He fiercely strongly denied those claims, accusing "cowards in the shadows" of trying to get at him through his wife and telling his critics they had "picked the wrong fight".
But as speculation continued that 25 Tory MPs would write calling for a leadership contest, he himself joined the band of people calling for a vote. He realised that an end was needed - whatever the outcome - to the plotting and sniping.