Tesla's figures concerning safety of autopilot are skewed
The statistics that Tesla provided in the wake of the recent Tesla Autopilot fatality are questionable, according to a statistician at the University of South Carolina. Tesla’s figure of one fatality after 130m miles compared its Autopilot crash rate to the overall US traffic fatality rate, including every mode of transport from pedestrian to 18-wheeler. However, the only valid comparative fatality rate benchmark for the Autopilot Tesla is the non-Autopilot Tesla. Human-piloted Teslas have been responsible for six crashes, one every 470m miles, while Autopiloted Teslas one in every 220m miles.

A couple of weeks ago, I wrote about Tesla’s claim that its Autopilot driver-assistance software is safer than a human driver.

After a fatal Autopilot crash last May, the company said the death was the first in 130 million miles of Autopilot driving—and noted that, “among all vehicles, in the U.S., there is a fatality every 94 million miles.”

The clear implication: Autopiloted Teslas are safer than human-piloted cars, and lives would be saved if every car had Autopilot.

DON'T MISS: How safe is Tesla Autopilot? Parsing the statistics (as suggested by Elon Musk)

But Tesla’s statistics are questionable at best. The small sample size—one crash—makes any calculation of Autopilot fatality rate almost meaningless.

Furthermore, Tesla compared its Autopilot crash rate to the overall U.S. traffic fatality rate—which includes bicyclists, pedestrians, buses, and 18-wheelers. This is not just apples-to-oranges. This is apples-to-aardvarks.

One statistician called Tesla’s comparison “ludicrous on the face of it.”

Tesla Autopilot Test

A better yardstick for comparison is the fatality rate for U.S. drivers of cars and light trucks compiled by the Insurance Institute for Highway Safety.

By that yardstick, the Tesla Autopilot driver fatality rate is almost four times higher than typical passenger vehicles.

But even that comparison is suspect because it measures the Autopilot Tesla against vehicles as disparate as the hulking Ford F-150 and the tiny Smart ForTwo.

In the end, I concluded, “…perhaps the only valid comparative benchmark for the Autopilot Tesla is the non-Autopilot Tesla.”

CHECK OUT: Tesla-owning pilot weighs in on use of term Autopilot

So let’s do that comparison.

Seven Model S fatalities

“Bubslug,” a writer for the stock market website Seeking Alpha, recently compiled a list of fatal Model S crashes in the U.S.

According to his list, there have been six in all, including the May Autopilot crash in Florida.

Tesla Autopilot suite of features - with version 7.0 update

Just last week, however, a seventh fatal crash occurred in Indianapolis. To date, there’s no suggestion that Autopilot was engaged before that crash.

So, in the U.S., that’s a total of one Autopilot and six human-pilot fatal crashes.

To arrive at comparative fatal crash rates, simply divide by the total miles driven by each kind of pilot.

ALSO READ: Tesla Autopilot 8.0 upgrade would have averted fatal crash, Musk says

Conveniently, Tesla itself has provided just those numbers.

On October 7, Elon Musk tweeted “Cumulative Tesla Autopilot miles now at 222 million.”

Earlier this year, Tesla announced that total Tesla miles driven had surpassed the two-billion mile mark, and published a chart projecting future total mileage.

Tesla Autopilot

According to that chart projection, total Tesla miles driven through September 2016 were predicted to be right at three billion miles.

We’ll accept that figure, with the caveat that it presumably includes European and Chinese cars. (We’re counting only U.S. incidents, remember.)

But we believe this discrepancy applies to both Autopilot and human-pilot cars in roughly equal measure, and should thus cancel each other out.

So, using round numbers, let’s do the math.

READ THIS: Tesla upgrades self-driving sensors, hardware; full autonomy test next year?

Human-piloted Teslas: six crashes, 2.8 billion miles. That’s one every 470 million miles.

Autopiloted Teslas: one crash, 220 million miles. One every 220 million miles.

And there it is, in hard numerical data.

Conclusion: Autopiloted Teslas have twice the fatal-crash rate of human-piloted Teslas.

Tesla Enhanced Autopilot

Still not statistically significant?

Clearly, the sample-size ogre raises its ugly head in this calculation: any statistic with a sample size of one will have a huge margin for error.

But these numbers, while not definitive, are far more meaningful than Tesla’s original comparison.

That’s because at least the comparative yardstick—Autopilot Teslas vs human-pilot Teslas—is reasonably valid.

If Tesla continues to throw around Autopilot crash statistics, it should at least get the comparative yardstick right.

Based on this more accurate comparison, Autopilot safety appears to be a far murkier and more troubling subject than Tesla has been letting on.

_______________________________________

Follow GreenCarReports on Facebook and Twitter.