Self-Driving Cars May Hit People With Darker Skin More Often, Study Finds (futurism.com)

According to a new paper from the Georgia Institute of Technology, autonomous cars could disproportionately endanger pedestrians with darker skin, a troubling sign of how AI can inadvertently reproduce prejudices from the wider world. Futurism reports: [In the paper, the researchers] detail their investigation of eight AI models used in state-of-the-art object detection systems. These are the systems that allow autonomous vehicles to recognize road signs, pedestrians, and other objects. They tested these models using images of pedestrians divided into two categories based on their score on the Fitzpatrick scale, which is commonly used to classify human skin color. According to the researchers' paper, the models exhibited "uniformly poorer performance" when confronted with pedestrians with the three darkest shades on the scale. On average, the models' accuracy decreased by 5 percent when examining the group containing images of pedestrians with darker skin tones, even when the researchers accounted for variables such as whether the photo was taken during the day or at night. Thankfully, the researchers were able to figure out what was needed to avoid a future of biased self-driving cars: start including more images of dark-skinned pedestrians in the data sets the systems train on and place more weight on accurately detecting those images.

Comments

Wrong

[SuperKendall]

This is not true at all, it's based on false assumptions.

First of all, most self driving cars will end up using LIDAR. Skin color, not an issue.

Secondly. even cars with cameras do a lot of image transformations such that color is usually disposed of. You kin color is irrelevant to a recognizer looking for human forms.

In fact you could argue that during the day, darker skin is an advantage because against a blue sky it's more noticeable than really pale skin which could look like clouds... #GingerLivesMatter.

Re:Wrong

[dgatwood]

This is not true at all, it's based on false assumptions.

First of all, most self driving cars will end up using LIDAR. Skin color, not an issue.

Secondly. even cars with cameras do a lot of image transformations such that color is usually disposed of. You kin color is irrelevant to a recognizer looking for human forms.

In fact you could argue that during the day, darker skin is an advantage because against a blue sky it's more noticeable than really pale skin which could look like clouds... #GingerLivesMatter.

Yes and no. Image recognition tends to be more sensitive to texture than to shape, and darker skin results in less contrast, which means less ability to see things like facial features that otherwise might identify the object as a human.

You are correct that object detection should not be a meaningful part of your strategy for avoiding hitting things. Rather, object detection is for doing things like traffic light detection, road sign reading, and determining where nearby cars are located so that you can calculate when to change lanes, whether you need to accelerate while doing so, etc.

Similarly, object detection should not be used for verifying that nothing is beside you, behind you, or in front of you. Those additional sanity checks are what RADAR, LIDAR, and SONAR are for.

Moreover, even if we assume that image recognition is used for that purpose, parallax differences between cameras should tell you that there is something in front of you. No matter how dark your skin is, if the car thinks that you're part of the road, the software is doing something very wrong, and it's the procedural part of the code base that is failing, not the image recognition part. After all, if dark skin is indistinguishable from the road, so are grey or black automobiles.

But — and this is a big but — detecting people near the road is often useful in terms of avoiding unexpected interactions later by slowing down, changing lanes, etc. And detecting gestures of police officers or other personnel directing traffic also needs to work regardless of their skin color. So it is important to ensure that training data doesn't show racial bias. The same is true for gender bias, attire bias, and any number of other things that could cause confusion for machine vision.

What bugs me about this article is not that the premise is wrong, because it isn't necessarily, but rather that it appears to be entirely built upon a giant tower of hypotheticals, such as the training data being inadequate, the computer vision being used for critical behavior rather than LIDAR or other tech, etc., none of which are necessarily going to happen in the real world, and all of which are readily avoidable by just not cutting corners in development.

Basically, it's like saying that a new nuclear reactor could seriously screw up the world if you forget to connect it to a water supply. My response is, "Yeah, no kidding."