It seems like Waymo's hitting a point of diminishing returns. The past 4 years' worth of disengagements per 1,000 miles driven were:
- 2014 (last 3 months): 1.27 [1]
- 2015: 0.80 [2]
- 2016: 0.20 [2]
- 2017: 0.18 [3]
There are two possible causes off the top of my mind:
1. Waymo's cars are being tested in more demanding environments.
2. There's a floor to the number of disengagements due to technology and road conditions: situations that are different/unique enough that no model could accurately detect/respond to it.
Case 2 would be definitely be a problem for the full-autonomous (i.e. zero user-input) type of car. I'd love to hear other hypotheses/feedback from those closer to this kind of work!
Also, another interesting note: total miles driven on public roads is actually down from 2016 (352k vs. 636k earlier). Perhaps Waymo's finding the simulated/private-road tests more fruitful for winnowing down the final edge cases.
Waymo is doing a lot more testing outside California now. They've done Mountain View so thoroughly they were seeing the same situations over and over. Last year, they were rear-ended twice at the same intersection.
One disengagement every 5000 miles is a problem. It's too infrequent for humans to remain attentive, and too frequent to ignore.
I think the devil's in the details a bit here. For example, for the first 251k miles driven in 2017 (71% of the total) they had 1 disengagement due to a "software discrepancy."
Then over the next 43k miles they had 8 disengagements due to software.
And then none since October.
Looks like a regression bug to me.
Similarly with "perception discrepancies" they had 14 disengagements for the first 175k miles driven (50%) and then 2 over the second half.
Also only 24 of the 63 incidents were "failure detections" where the vehicle immediately handed over control of the vehicle; the rest were handed over "safely" (I assume this means the driver took over control voluntarily.)
Also they don't say what the criteria is, but I suspect given what we've heard about the cars' conservative natures that "disengage for unwanted maneuver of the vehicle" is code for "the car is going too slow / being too passive at an intersection, etc."
So all in all I see positive trends above and beyond the overall numbers. But I'm bullishly biased about self-driving cars, so get out your salt grains.
- 2014 (last 3 months): 1.27 [1]
- 2015: 0.80 [2]
- 2016: 0.20 [2]
- 2017: 0.18 [3]
There are two possible causes off the top of my mind:
1. Waymo's cars are being tested in more demanding environments.
2. There's a floor to the number of disengagements due to technology and road conditions: situations that are different/unique enough that no model could accurately detect/respond to it.
Case 2 would be definitely be a problem for the full-autonomous (i.e. zero user-input) type of car. I'd love to hear other hypotheses/feedback from those closer to this kind of work!
Also, another interesting note: total miles driven on public roads is actually down from 2016 (352k vs. 636k earlier). Perhaps Waymo's finding the simulated/private-road tests more fruitful for winnowing down the final edge cases.
[1] https://www.dmv.ca.gov/portal/wcm/connect/dff67186-70dd-4042...
[2] https://www.dmv.ca.gov/portal/wcm/connect/946b3502-c959-4e3b...
[3] https://www.dmv.ca.gov/portal/wcm/connect/42aff875-7ab1-4115...