Traditionally, skin tone distortion in computer vision is measured using the Fitzpatrick scale, which ranges from light to dark. The scale was originally developed to measure the tanning of white skin, but has since been widely used as a tool to determine ethnicity, says William Thong, an AI ethics researcher at Sony. It is used to measure bias in computer systems, for example by comparing how accurate AI models are for people with light and dark skin.

But describing people’s skin on a one-dimensional scale is misleading, says Alice Xiang, Sony’s global head of AI ethics. By classifying people into groups based on this rough scale, researchers avoid biases that affect, for example, Asian people, who are underrepresented in Western AI datasets and can fall into both light-skinned and dark-skinned categories. And it also doesn’t take into account the fact that people’s skin tones change. For example, Asian skin becomes darker and yellower with age, while white skin becomes darker and redder, the researchers point out.

Thong and Xiang’s team developed a tool — shared exclusively with MIT Technology Review — that expands the skin tone scale into two dimensions, measuring both skin color (from light to dark) and skin tone (from red to yellow). Sony makes the tool available online for free.

Thong says he was inspired by Brazilian artist Angélica Dass, whose work shows that people from similar backgrounds can have a wide variety of skin tones. However, representing the full range of skin tones is not a new idea. The cosmetics industry has been using the same technology for years.

“Anyone who has ever had to choose a foundation shade knows how important it is not only whether a person’s skin tone is light or dark, but also whether it is a warm or cool tone,” says Xiang.

Sony’s work on skin tone “provides insight into a missing component that people have overlooked,” says Guha Balakrishnan, an assistant professor at Rice University who has studied bias in computer vision models.

Measurement error

There is currently no consistent way for researchers to measure bias in image processing, making it more difficult to compare systems with each other.

To make bias assessments more efficient, Meta has developed a new method for measuring fairness in computer vision models called Fairness in Computer Vision Evaluation (FACET), which can be used for a number of common tasks such as classification, detection and segmentation. According to Laura Gustafson, AI researcher at Meta, FACET is the first fairness assessment that spans many different computer vision tasks and includes a broader range of fairness metrics than other bias tools.