More than 60% of marketers have utilized artificial intelligence (AI) in their marketing activities so far. Marketing relies on data-driven insights and innovative technologies to connect with our target audiences. But what happens when those very technologies inadvertently reinforce biases and stereotypes? How can marketers ensure that the use of artificial intelligence remains ethical, transparent, and inclusive in an ever-evolving digital landscape?

AMA SF spoke with Steve Anderson, a Bay Area-based AI expert and a 20-year veteran in the data industry, to learn more.

Why diversity is essential for successful AI use

AI models are trained on massive amounts of data. In the case of popular tools like OpenAI’s ChatGPT, the model was trained on publications, books, and quite literally the entire contents of the internet.

“I’ve seen amazing things with AI — I’ve seen lives improved, I’ve seen the government decrease healthcare costs, and I’ve seen businesses increase their revenue,” Anderson said. “There are so many positive things coming out of AI… as long as we’re doing it in a responsible and trusted way, I think we’re going to continue to make strides.”

While every model has its own nuances, it is imperative that AI models are fed or trained with “trusted” data. This refers to the data’s quality, integrity, and completeness necessary for positive outcomes.

“Trusted, in this context, means that someone made sure that the data is complete and that the model captured the data in a way that’s error-free,” Anderson said. “For example, you may encounter data with missing values, and you have to decide what to do with those missing values.” Overall, Anderson said Trusted data means the data is accurate, has integrity, high quality, transparency, minimizes bias, and respects ethical concerns.

If the data being used is missing diverse points of view or information on diverse populations, the chances of skewed results increase. This could lead to marketers inadvertently working with AI model outputs that don’t reflect the true nature of their target markets.

“Diversity is so important here — we need those different perspectives for a balanced experience,” Anderson said. “We have customers, consumers, constituents, and stakeholders that look like all of us, so all of us need to be represented in the data AI is using… that brings humanity to this space.”

Bias, racial bias, and the way both show up in AI models

Bias in AI refers to any process or result that suggests a discriminatory outcome that doesn’t reflect the underlying reality. Generalization and oversimplification are two examples of biased AI results.

“There are a lot of different biases that can happen in this space, and it’s just as important that practitioners are aware of those different biases so that they’re thinking about it consciously as they prepare their data,” Anderson said.

Anderson said that while bias in machine learning is often “unintentional,” the ramifications can have significant consequences on Black communities, people of color (POC), and other underrepresented groups. He underscored that bias can inadvertently infiltrate various stages of the AI initiative – spanning from the planning and data preparation phases to the training, which could even unintentionally introduce racial bias into the AI model. Anderson cautioned marketers using AI tools to remain vigilant about this possibility.

While there are many pathways and models that could contribute to bias in AI, Anderson called attention to three to help illustrate how these issues of racial bias may occur. But he highlighted that there are more than three of them.

1. Lack of proper representation. This happens when there is a lack of representation in the dataset. In this case, the dataset does not have enough data or the right data, lacking the necessary breadth and relevance across multiple demographics.

“We’ve seen examples of this, especially with facial recognition and [similar use cases], where the model is going to make a prediction regardless of whether you give it enough information about an unrepresented group or not,” Anderson said. “Having that diverse perspective can flag these instances early.

”What is described above is a lack of representation from a data perspective. However, from a people perspective, a lack of representation amongst the people at the table implementing AI solutions can also potentially and unintentionally lead to biased outcomes.

2. Data outliers. These are data points that stand out from the rest of a set which can significantly skew data. Outliers can be caused by system errors from the data source, data entry errors, data processing errors, natural variation, extreme events, and many other reasons. Anderson said data outliers can come from a number of sources, all of which can contribute to biases in the final results.

“It’s important to know if something in your data… doesn’t make sense before it’s used,” Anderson said.

3. Selection bias. If the data used to train an AI model does not represent what’s happening in the real world, that’s an instance of selection bias. This could happen due to incomplete data, non-random sampling, and convenience sampling, among other ways.

“Some practitioners may train machine learning algorithms on an entire body of data, while others may do a random selection,” Anderson said. “If the random selection wasn’t done properly and it doesn’t totally represent the entire population, that can create bias as well.”

4. Exclusion bias. This occurs when data is deleted that would have otherwise added value to the AI model being trained. Anderson provided the example of removing a percentage of a customer list with missing data. The missing data may have been meaningful enough to influence outcomes, causing skewed results.

“When you train a Large Language Models (LLMs) model [like ChatGPT], it’ll train on what you give it,” Anderson said. “If something important is missing, it can have an impact on what it predicts.” Furthermore, Anderson said that LLMs do not know if the data it is trained on is accurate or not; therefore, it is equally important that LLMs are trained with accurate data only.

High-profile examples of racial bias in AI models

While the conversation around racial bias in AI is not a new one, the release of ChatGPT to the public in November 2022 has brought many of these conversations to the mainstream.

Several high-profile examples of racial bias in AI models include:

• A study published in April 2023 found that ChatGPT produced “incorrect stereotypes, harmful dialogue, and hurtful opinions” when asked to filter results through certain personas, or points of view.
• A New York Times article in July 2023 reported findings of racial bias in artificial intelligence, stating that several AI tools struggled to recognize the facial features and speech patterns of non-white people.
• In April 2023, a Forbes contributor article noted multiple examples of facial recognition technology that had racial bias built into the algorithm, consistently classifying people of color as criminals and struggling to discern Black faces altogether.

4 ways marketers can minimize racial bias in AI

“The overall goal is to have as minimal bias as possible,” Anderson advised. “And to do that, you really have to think about [diversity] in a broad way.”

1. Promote transparency and accountability

More voices in the room can help reduce bias and help catch biases early on.

“You need a lot of different perspectives when you do projects like this, which means you need to bring a lot of people to the table,” Anderson said.

Anderson recommends putting together a governance committee that oversees AI use at your company. This committee plays a central role in ensuring AI models are used in an ethical and inclusive manner, while a diverse body helps catch any issues that may slip through the cracks.

“This diverse group of people [on a committee] all have different perspectives, and they can use their collective backgrounds to vote yea or nay on an initiative,” Anderson said. “The committee can be a really important, objective body influencing how the model is working and improving.”

Ideally, Anderson said, the committee would maintain a list of issues to track, evaluate, and recommend changes if needed. The committee also keeps up with what is happening in the market.

“You may see something come out in the news [about AI] and realize that your organization is doing the same thing, for example, and you could pose the question if it’s right for your company to keep going on a similar project,” Anderson said.

2. Promote transparency and accountability in the process

Anderson recommends partnering with the practitioner who builds and trains AI models to establish your goals and provide the necessary data to achieve those goals accurately.

“Instead of letting a practitioner go off and build something… go along with them on the journey, all the way from defining objectives and KPIs… all the way to testing,” Anderson said.

He also recommends using AI tools that have built-in guardrails to help detect issues early on.

“You want to make sure you have the necessary reporting and dashboards to have transparency on what is going on with your models,” Anderson said. “For example, some platforms have tools that can help you model operations and others with monitoring tools so you can better track what the models are doing.”

3. Put in effort to ensure data is trusted and complete

Datasets are not complete if they leave out large swaths of the population, Anderson said. No matter how your organization uses AI, Black and POC representation minimizes the possibility of biased results.

“The system doesn’t know [about racism]; it only knows what it was trained on. You got to be mindful of that in your data,” Anderson said.

4. A/B test campaigns and creative that use AI

Anderson said A/B testing is key to “minimizing any particular risk,” adding that it’s necessary to “do right by customers and consumers” by putting yourself in their shoes and assessing your work from their perspectives.

“Think about the end user — there’s someone on the other side who will receive the messages you’re shaping with AI,” Anderson said.

Conducting small tests before fully deploying an AI-informed initiative can help catch issues before they become larger problems.

“Don’t deploy results from AI across your entire business right away — start with a little subset [of the initiative] and work with multiple teams to design an A/B test that can help catch any issues,” Anderson said. “I recommend that these teams are cross-functional. For example, you most likely have a customer service team, a marketing team, a data science team, and some type of technology team. That cross-functional, diverse team comes to the table to help with the A/B testing.”

Put AI to work in your organization in a smart and thoughtful way

Artificial intelligence is here to add value to humans, not make life more complicated. As AI continues to change how marketers do their jobs, understanding how these models work — and pitfalls like racial bias that may occur due to the way the models work — ensure that your organization implements AI models’ output in a responsible way. Including diverse viewpoints from the onset can help minimize these occurrences and flag them as they begin to emerge. Putting these safeguards in place ensures a better outcome overall.