4 Cognitive Biases In AI/ML Systems

Author(s): Lye Jia Jun

Originally published on Towards AI the World’s Leading AI and Technology News and Media Company. If you are building an AI-related product or service, we invite you to consider becoming an AI sponsor. At Towards AI, we help scale AI and technology startups. Let us help you unleash your technology to the masses.

In the ideal world, machines are unbiased and objective. However, the humans who engineer these machines are inherently biased.

With the rapid proliferation of advanced technology, more and more systems are now equipped with artificial intelligence and machine learning algorithms.

However, can we objectively say that these systems are truly fair and unbiased?

In this article, I share 4 cognitive biases that AI/ML systems have and how machines could be full of biases because the engineers who built them are inherently imperfect.

Selection Bias

• Selection bias refers to the selection of training/testing data that is not representative of the entire population.
• Example: An engineer chooses the first 100 volunteers who responded to his email as his training data
• The problem: The first 100 respondents may be more enthusiastic about a product or study than the last 100 respondents. By explicitly choosing the first 100 respondents, the engineer introduces unfairness in his data collection method.
• The solution: select a random sample of 100 users from your pool of email respondents instead of the first 100.

Reporting Bias

• Reporting bias refers to people’s (conscious and unconscious) tendencies to suppress the information they report.
• Example: Many amazon products have more 5-star and 1-star reviews than 2, 3, or 4-star reviews because people who have extreme experiences (either positive or negative) are more likely to post a review than those who have neutral experiences.
• The problem: An engineer that uses online reviews as their primary data source may create an AI model that is great at detecting extreme sentiments but not so much at detecting more neutral, subtle sentiments.
• The solution: Consider broadening the data collection scope to account for the underrepresented data.

Implicit Bias

• Implicit bias refers to people’s unconscious tendencies to make assumptions or associate stereotypes with others.
• Example: An engineer who is often bitten by dogs believes that dogs are more aggressive than cats, even though that may not be scientifically true.
• The problem: The engineer would believe that the ground truth is “dogs = aggressive” and thus fine-tuned her AI model to label dogs as being more aggressive than cats.
• The solution: As implicit bias is unconscious to an individual, having multiple engineers code the AI and establishing proper peer review procedures would reduce the occurrence of such biases.

Framing Bias

• Framing bias refers to people’s tendency to be influenced based on how information is presented.
• Example: An engineer who sees a dark and dull website for a product believes that the product must have poor sales, ignoring the actual positive sales number of the product.
• The problem: In designing the AI algorithm, the engineer may take into account subjective variables, such as the color of a website, instead of focusing on objective metrics.
• The solution: Avoid subjective (usually qualitative) data and prioritize objective, factual data instead.

Summary and Best Practices

A quick recap of the 4 cognitive biases in AI/ML systems

• Selection Bias (selecting data not representative of the entire population)
• Reporting Bias (people’s tendency to underreport information)
• Implicit Bias (people’s unconscious tendencies to assume)
• Framing Bias (people’s tendency to be affected by how information is presented)

The 5 best practices to enhance objectivity in AI/ML systems

• Always select a random sample (instead of the first or last hundred data points)
• Verify your data sources through comparison with other data sources
• Assign more (diverse) engineers to develop the AI/ML system
• Establish proper peer review procedures to cross-examine logic and unconscious bias
• Prioritize objective and factual data over subjective (usually qualitative) data.

As you can see, humans’ perceptions are deeply flawed and our imperfection may trickle down into the systems we build. By acknowledging such biases, however, we can optimize the AI systems we build and make them more fair and objective.

I hope you learned something new today. If you like what you’re reading, do drop a clap or a follow!

I’ll catch you in the next article. Cheers!

4 Cognitive Biases In AI/ML Systems was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Join thousands of data leaders on the AI newsletter. It’s free, we don’t spam, and we never share your email address. Keep up to date with the latest work in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI