Dave Stewart replaces Jonathan Bacon who stood down citing a lack of support he said made his position "intolerable".
Independent Ian Stephens and Conservative Dave Stewart were both nominated.
Mr Stewart received 19 votes, while Mr Stephens received 15.
Mr Bacon, who led the council for two years despite his group having no overall majority, was not at the meeting to choose his successor.
Deputy leader Steve Stubbings, who abstained in the vote, and another member of the island council's six-person executive, Gordon Kendall, also announced their resignations.
In a resignation statement, Mr Stubbings and Mr Bacon said the "damaging behaviour" of Isle of Wight councillors was "overwhelming" efforts to deal with government cuts.
Mr Bacon said: "We now have a situation amongst elected members where party politics and ego are being put before the concerns and issues of island people."
Isle of Wight Council declined to comment.
In October, Mr Bacon reversed a council vote against devolution, saying a Â£900m deal to set up a new authority for the Solent area was the only way to address the authority's "perilous financial state", following budget cuts of more than Â£50m over the previous five years.
A Conservative has been selected to lead Isle of Wight Council following the resignation of its Independent leader on Monday.