Relevant short and long-term memory of every 3d object you've ever seen along a road-side and what the expected behaviour of those objects should be. Does a road-sign usually move by itself? No, maybe someone is carrying it. Is a ball coming from behind a car in a school-zone likely to be chased by a child...
> Is a ball coming from behind a car in a school-zone likely to be chased by a child.
Yeah, that kind of agent prediction is a biggie. Agent prediction is something journalists don't talk about much, but is a huge research topic for AV practitioners.
And since you make note of expected behavior of objects... my all-time favorite bug report out of Waymo (that made the public news) is a classic classifier/predictor/planner corner case leading to deadlock. Classifier identified a cyclist and labeled as "stopped" if cyclist has foot on the ground, and "moving" if both feet on pedals. Predictor would plot a trajectory for moving cyclist, planner decides action based on predicted agent behavior. So... Waymo vehicle and cyclist on two corners of 4-way stop. Cyclist is stopped, doing a perfectly balanced track stand with both feet clipped in. Feet on pedals so cyclist is labeled as moving agent... planner yields right-of-way to cyclist according to rules-of-the-road. Deadlock -- nobody moves.
Isn't that how Waymo killed a pedestrian/cyclist? It couldn't decide if she was a cyclist or pedestrian, as she was walking and pushing her bicycle, so it never braked.
She was homeless so the shape of her bike wasn't a proper bike so it got confused.
Yes, Uber, operating in Level 3, and safety driver was watching a movie on phone at the time.
Very different circumstances. I don’t think this was a classifier defect. Not that I was ever a fan of Uber’s development program, but in this case I can’t fault the platform.
