The system could switch to manual or more cautious autonomous driving when a confidence score is not met. The confidence score could be a score based on how many other cars have driven the road, number of accidents, satellite footage, etc.