Philip Garvey, 56, his wife Ann, 55, and their children Emily, 23, and Daniel, 20, died when their plane came down in Somerset in November 2015.
They were flying from Surrey to Devon.
The Air Accidents Investigation Branch (AAIB) said it was possible Mr Garvey tried to manually start landing while the autopilot was still on.
Mr Garvey had taken off from Fairoakes Airport, close to the family home in Woking, and was heading to Dunkeswell Aerodrome.
The AAIB's report said it was an overcast and rainy day and all planned flights at Dunkeswell had been cancelled because of the weather.
Several witnesses told investigators they heard a sound like a plane doing "aerobatics" before the Piper Malibu Mirage appeared from the clouds and hit the ground near Churchinford, south of Taunton.
The AAIB said: "The evidence from the autopilot examination system suggested that, as the pilot turned on to the final approach and started to descend, the autopilot may not have been disengaged due to a mental lapse, incorrect button selection or a technical fault."
The plane was seen to quickly climb into cloud before coming down.
The AAIB concluded it was "unable to determine with certainty the reason for the initial rapid climb" but said attempting to override the autopilot would have caused the plane to go "nose up" as it would fight to maintain its set height.
While the aircraft could still have been controllable, it said "recovery may have been beyond his capabilities".
Mr Garvey had been a pilot since 2012 and had flown to Dunkeswell 25 times since 2013, including 14 times from Fairoaks.
His instructors told investigators he was "less able than average pilots at multi-tasking and poor at prioritising especially when under pressure".
Another instructor said Mr Garvey "would often have difficulty dealing with an unusual or unexpected condition".

The opposition said it would work to release jailed opposition leaders and address the country's economic crisis.
Mr Maduro has announced a cabinet reshuffle but vowed to veto any amnesty law for jailed politicians.
The election result is the worst for the Socialist movement founded by late leader Hugo Chavez in 1999.
The majority means the Democratic Unity coalition can now make sweeping changes, including replacing Supreme Court judges and making changes to the constitution pending approval in a referendum.
It also wants the release of one of its leaders, Leopoldo Lopez, who was given a 13-year prison sentence for inciting violence - a charge critics say was politically motivated.
But Mr Maduro said he would block any amnesty law.
"The murderers have to be prosecuted and have to pay," he said in a televised address.
He said his Socialist Party would hold an "extraordinary congress" to find out what went wrong at the election.
The opposition meanwhile warned of looming food shortages for Venezuelans.
"We urge the government to stop crying and start working," opposition leader Jesus Torrealba said.
"We're just a few weeks away from a very serious problem in terms of food," he added.
The elections were widely seen as a referendum on President Maduro, the handpicked successor of Hugo Chavez, and the governing PSUV's socialist policies.
The opposition accused Mr Maduro's party of mismanaging the economy and of squandering the country's oil wealth.
Venezuela has been hit hard by the continuing low price of oil, its main export. It also has the continent's highest inflation rate.
Mr Maduro has blamed the situation on an "economic war" waged by the opposition.
The opposition also accused the government of increasing authoritarianism.
The election result also gives stronger momentum to the opposition should it wish to call a referendum on Mr Maduro's future. This could take place only when his presidency reaches its halfway point in April next year.
However, some prominent opposition figures want to give Mr Maduro the opportunity to change policies they blame for the country's economic crisis.
"If Maduro doesn't change we'll have to change the government. But the opposition's response to the economic crisis right now can't be more politics," Henrique Capriles - who lost to Mr Maduro in the 2013 presidential elections - told AP.
The next presidential election is due in April 2019.
100%+
inflation level for 2015
-10% shrinking of GDP in 2015
-6% projected 2016 downturn
18.1% projected 2016 unemployment rate
A pilot who died along with his family when the plane he was flying crashed was "poor under pressure", his instructors have told investigators.
[NEXT_CONCEPT]
Venezuelan electoral authorities have confirmed that the opposition has won a key two-thirds majority, enabling it to challenge President Nicolas Maduro.