Vehicles with advanced driving assist systems that automatically steer, accelerate and brake are popular, but associated with increased driver distraction. This distraction coupled with unreliable autonomous system performance leads to vehicles that may be at higher risk for striking pedestrians. To this end, this study tested three consumer vehicles in two different model classes in a pedestrian crossing scenario. In 120 trials, one model never detected the pedestrian, nor alerted the driver. In 123 trials, the other model vehicles almost always detected the pedestrian, but in 35% of trials, alerted the driver too late. These cars were not consistent internally or with one another in pedestrian detection and response, and only sparingly sounded any warnings. These intelligent vehicles also detected the pedestrian earlier if there were no established lane lines, suggesting that in well-marked areas, typically the case in for established crossings, pedestrians may be at increased risk of a possible conflict. This research demonstrates that artificial intelligence can lead to unreliable vehicle behaviors and warnings in pedestrian detection, potentially catching drivers off guard. These results further indicate industry needs to do more testing of intelligent systems, regulators should reevaluate the self-certification approval process, and that more fundamental work is needed in academia around the performance and quality of technologies with embedded neural networks.
I feel like the primary problem here is just that detecting pedestrians and figuring out how they are going to is actually one of thouse problems that is just very hard for computers. Obviously it’s not impossible to do at all, but it is difficult to do reliably, especially once one considers the risks of false positives as well as negatives.
It is definitely concerning though that these systems are being pushed and marketed beyond their actual capabilities though. Some proper truth in advertising law might actually help here, and of course in an ideal world this would be an open source project with all the major automakers and academics contributing.