Depending on where you set your standards, sure. But we have an expectation in the United States safety technology has to accommodate darn near everyone.
Now sure, a passenger can be different than the driver but it's the same philosophy.
The amount of illegal maneuvers I see every day on my commute is astonishing - not using blinkers, intruding on cross walks, not moving over for emergency vehicles, following too closely, etc. It doesn't help that the only traffic law enforcement is really around speeding / running red lights / DUIs.
The problem with autonomous cars isn't the autonomous cars - it's accommodating non-autonomous actors. It only takes one google car hitting an old lady who's chasing a duck on the street to become CNN breaking news for the next 3 months a la that airliner that disappeared.
