How Ethical AI can redefine the role of global businesses

Welcome to Insight Connect, an initiative where we invite a range of local and international experts to share their experience, insights, and view on cultural intelligence and why it matters.

Today I am delighted to welcome Gabriela Del Barco-Renard, a DE&I consultant and facilitator with a focus on responsible and ethical AI. Together we are going to be discussing the role that businesses play in supporting responsible and ethical AI and the global societal impact it has. This happens through businesses shaping the narrative by implementing regulation, governance, and intentional design, because from cultural bias to exclusion, AI really has the power not only to connect, but also to divide and polarise.

https://www.youtube.com/watch?v=eg-x2ASbhc8

Transcript

int!: Welcome, Gabriela. It's a pleasure to have you with us today.

GDR: Thank you for that super introduction. I'm delighted to be speaking to you, with you today. So let's dive in.

int!: Brilliant. Thank you, Gabriela. I'm always fascinated to hear why people do what they do, so could you tell us a little bit more about how you landed at DE&I and cultural intelligence in the first place?

GDR: It's not linear at all. I actually started my career as a tax lawyer and I did that for six, seven years. But I knew that that wasn't something that I wanted to do for the rest of my life. So, I made a big shift. I left Bolivia where I originally come from. I moved to Germany to study intercultural conflict management, and that's what led me to the humanitarian world. I worked for the UN for several years in humanitarian coordination and gender equality. So, very complex and very multicultural settings. That really was a learning experience. It was really rich in terms of how diverse teams work together and come to work together.

And then later, I moved to France and I became a migrant myself. And so I lived the barriers and the struggles that come with that title and the things that we actually see or hear about in theory, I actually lived them, and that's what actually led me to the DE&I space. So first working for Singa, an organization that connects locals with refugees and asylum seekers to create opportunities to connect with each other and also to change the narrative around migration and asylum.

And eventually I founded my own consultancy, it's called Collective Studio. And early on, I started to become interested in the connection or the intersection between DE&I and AI. And it actually started with a conversation with a computer scientist who at the time was working on AI fairness. And he said: " We are supposed to be building these fair AI systems, but we actually don't have the knowledge that's required from the social science perspective. We don't know what social exclusion looks like, et cetera". And so that was a wake up moment. And so I started like learning a lot. I joined AI ethics courses and started to building these bridges between these two disciplines. And so that's how I do my work today. I help organizations build inclusive cultures, but also to take this same perspective in the way they use and deploy AI. That's pretty much my story.

int!: Amazing. It's really fascinating. Very inspiring background. And in fact, expanding on this, we had this conversation before and you are a strong advocate for involving non-technical profiles in developing AI solutions and tools, believing it is not a conversation only for data scientists and engineers, but there is a need to involve more soft skill experts, people like you working in the cultural intelligence space to really help support how we build AI within large businesses (or smaller)'s ways of working. Can you tell us a bit more about that and some examples you may have encountered in your career?

GDR: I think the first thing we need to understand is that AI is not just a technical system, it's a social technical system. So, it's not only fueled by algorithms, but it also learns from data that actually comes from us as individuals and as societies. And it's also shaped by the social context in which it is developed and deployed. And so that brings with it all the biases and inequalities and power relations and everything that we have in our society. So, understanding the math or how to build an AI system is just one part of the story. We need to really acknowledge this interaction between the social and the technical in that sense. And the other thing that we need to understand is that technical systems, technical teams are often trained for things like accuracy, efficiency. So they optimize these kinds of things and when they work in silos, what's often missing is the ability to ask different questions. What are we actually measuring? Who might be excluded or harmed by our systems? What type of assumptions are we embedding into the systems without even realizing it?

So, it's not just a question of: is this system technically robust enough? But how is it impacting people and society? And that's where the real input from social sciences and humanities. comes into place. So let me give you an example of what it looks like when we leave this knowledge behind or leave it aside.

There's a well-known study by Ziad Obermeyer and his colleagues at Berkeley. They were analyzing this healthcare algorithm that was used on millions of patients in the US. The goal of the algorithm was to identify patients who needed additional care. And to simplify the problem, developers used healthcare spending as a proxy for health needs because it's easier to measure and because there is data, around that. But of course, because of unequal access to the healthcare system, black patients and other underserved communities tend to spend less, even when they are as equally sick. So, at the end of the day, what the algorithm was doing was systematically underestimating their needs. And as a result, black patients were less likely to be referred for additional care than equally sick white patients. So technically, the algorithm or the AI system worked. But actually, what was happening was that it was reproducing and amplifying existing inequalities. And that's exactly where expertise from people like us, from the social sciences and humanities and DE&I experts and culture experts, we have the knowledge and a way of framing issues that is different.

We ask different questions. We are going to look at power dynamics. We're going to see how categories are constructed. We ask questions about how the systems are interacting in the real world. And so the thing for me is not to oppose the technical to the non-technical, but really to bring the right conditions to make interdisciplinary collaboration work, and we are starting to see how this works well in reality.

I will give you another example. When Adobe was developing the Firefly model (that's a text to image generator from Adobe), the technical team worked with the product equity team, ethical innovation, trust and safety and legal teams. So, they brought together this interdisciplinary team that integrated more easily diverse perspectives. They actually included also the voice of affected communities into the conversation. And at the end of the day, they created a product that is not just robust, but it's also, it's also more inclusive. It enhances trust by consumers, et cetera, so it's a better product overall.

int!: Amazing. This brings me to the reason I wanted to start intention!. I always say (CQ) it's the intelligence you didn't know you needed. And it's this sort of extra layer of due diligence that doesn't always occur. It's not just in technology, it is in a lot of internationalisation project. It's just having this additional perspective to understanding the reality, right? So, to your point, there's not one answer, or there's not just one question, but there is a reality, and it's about trying to see that granularity and that nuance that that exists. Speaking of which, we've discussed recently together, offline, how AI models have been pretty much trained on Western data, which has in turn a huge implication on the output, but also in how we shape users' view of the world in general. We discussed the really interesting study from restofworld on this, and I was wondering how you feel. Well, firstly, tell us a bit more about that study. How do you feel this data is affecting the underrepresented communities, and how can we address this? How can businesses make sure that this is tackled?

GDR: This is one of the biggest questions in the responsible AI space. There's actually a name for that. Some researchers call it 'Western Gaze', and even 'Silicon Gaze'. So, it is the way AI systems and specifically generative AI systems tend to project Western norms and values, and then spread them into different global contexts. And this is not just about cultural nuance, it's also about spreading negative stereotypes. It's about, misrepresentation and even erasure of certain communities and certain parts of the world. And actually a big part of this problem comes from data, as you mentioned. Actually most of the data that is used to train AI systems comes from the internet. We scrap the whole internet and we fit it to the model and the model learns from it. And the vast majority of the data that is present on the internet comes from, it's in English first and it comes from the US and Western Europe. It's a majority of this data. Because we leave a bigger digital print, we're more connected, and so we leave more data there. So, the model's common sense, the way it interprets the world is shaped by these perspectives.

And then there's also the context in which the systems are built and the composition of teams, which are largely Western as well. We have an acronym for that: WEIRD. It's: Western, Educated, Industrialized, Rich and Democratic. And so these perspectives are embedded into the AI system without even realizing it. So, for example, we see that even models that are not trained in English can still have this bias. For example, there's this restofworld research on image generation. Researchers asked the model to generate images of things like a person or women or a plate of food in different countries, non-Western countries and they built up a data set of like 3,000 images. And what they found was pretty stereotypical outputs. For the Mexican person, you have almost 99% of men wearing kind of a sombrero or something like that. And most of images with a moustache. So it's really cartoonish. It's really stereotypical. It really made me think about Speedy Gonzalez, you know. And so what the model is doing is flattening the richness and the diversity of these cultures and to simplify often to really reductive representations. And what we see is that it will become a bigger issue because these systems are increasingly used in media, marketing and in education. And they shape how people see themselves and how we see other people. And that's a huge issue in terms of representation.

And some researchers now are arguing that this is not just an actual feature. It's not just a glitch, it's an actual feature of how the systems are built because they are trained on huge data sets that combine hundreds of years of a single story. And so addressing this requires more than just tweaking the system or adding more data. It requires governance, asking for more transparency from AI builders, which data is going in, et cetera. And it also requires from us users a different way of using them. One thing that I believe, and research also shows that it's really important, is how to include underrepresented communities and local communities, but in the good way of doing it. So, for example, there are initiatives, that are building these data sets on Quechua or Aymara or indigenous languages to build this knowledge and to integrate it into AI systems. So you need to do that to involve local communities, but also like to compensate them fairly and to use their data in an appropriate manner.

int!: So, when we look at the social impact more specifically, and again, everyone talks about businesses using AI for increased efficiencies and processes and so on and so forth. But again, there's a role for businesses to have in the conversation and making sure that we're not increasing the divide because research shows that half of the harms that AI is generating at the moment is related to social inclusion and all these sorts of topics because we're not able to accompany people who don't have education related to how to best use AI or may not have been helped with having a critical sense of the data they see in front of them. And knowing that well, Mexican people don't wear sombrero and aren't male, to your point. Even though I exaggerate this one, how do we help companies prevent this social gap from widening. What can they do and what's happening already? Is there some best practice that we can learn from and companies that are doing it right?

GDR: So I think we need to understand that AI has both the potential to act as an equalizer, and also to deepen existing inequalities. So it's not just one or the other, it's both. And we need to understand that framing. So on the one hand, AI actually can make access to skills and capabilities easier for certain communities.

And we see research showing that, for example, less skilled workers using AI can actually improve their productivity and kind of level up. And we also see how AI is helping people facing language barriers or disability. And we see also these great examples of AI used to improve access to education, financial inclusion, etc. So, there's a good side of this story, but the more we know, the more we see that AI is actually widening inequalities. And one thing that we need to frame is that AI doesn't exist in a vacuum. AI is entering a world, a society and organizations that are already unequal. So we need to understand this dynamic and how AI can perpetuate inequalities. And so the first thing is access, and it's not just about access to AI tools or infrastructure, so the digital divide, but also access to the right skills to use AI appropriately. So, even when tools are available, people don't have the same level of digital literacy to support their use effectively. The digital divide is both infrastructure, tools and education. And the second thing is the benefits are not even, so the equalizer effect I just mentioned only works for things that are more routine tasks or things that are related to customer service or administrative work.

But when we see that it hits a wall, it's for more complex or knowledge-complex tasks like for example, scientific research. And we see there that if you don't have sufficient experience, if you don't have enough critical thinking to actually assess the outputs of the AI system, you hit a wall and you can risk relying on incorrect or misleading information, and then the whole thing backfires. And then there's of course the question of job exposure and so many roles that are highly exposed to automation, like I said, administrative, work, customer service, etc. are often lower paid and disproportionately held by women and marginalized groups.

There's even data from the ILO showing how these disparities affect differently women than men. And we see that, for example, 16% of female dominated occupations are following the highest exposure categories compared to only 3% of male dominated jobs. So there's disparity to take into account as well. And so we need more deliberate intervention to prevent AI to widening these gaps. And so what does it mean the whole thing for organizations? The first thing is that we need to see more than AI being a tool for efficiency and productivity. Adoption needs to also be frame as transformation: how work is being transformed with the use of AI, who benefits from that transformation? And one of the things and the good experiences here is that when you involve employees into this conversation, so they can reimagine their jobs with the use of AI and you give them the right tools and the right setting to experiment, you have better results. So it's not something that is imposed to them, but the conversation is with them, you know?

And the second thing is that you need to invest in upskilling more equitably. Right now, we see that more senior roles have more budget and have more support for AI literacy training or AI training in general as opposed to entry level jobs and lower paid jobs. And also I think we need to reframe how we teach AI - it's not just about how to use AI, but also how to develop the skills that are necessary to use AI effectively. So we are talking about critical thinking, etc.

And the last thing we need to pay attention to is structural barriers in teams. So, for example, we see a persistent gender gap in AI adoption. So around the world, even when access is equal, women are 20% less likely to use AI tools. It's not just because there's less trust by women to into AI systems, but actually because out of fear of being perceived less competent than men, and there's research that showed that women data scientists that use AI to generate code were perceived as less competent than men, even when the outputs were equally qualitative. So again, that brings us back to the questions of culture, trust, organizational norms, etc. And I really, really believe there are organizations that have already more inclusive cultures that create the psychological safety and support, support experimentation where people actually can be heard and expose their fears and their questions around this transformation, will have better chances to use AI to enhance human capabilities instead of widening existing gaps.

int!: I really like your perspective. It's very rich. It is very deep and it covers a lot of areas. I think the positive is there is a real opportunity for upskilling within businesses, and they have a real role in bringing their employees onto that journey, obviously, but they really must understand their internal audiences, right? To your point, it's understanding: where are the gaps, where are the needs? Where do we support people to bring them on this growth journey altogether? Whilst always having the right elements of governance to make sure that we're not amplifying the negative, we're actually amplifying the positive and bringing everyone together.

It's interesting you mentioned that. I have read some of the studies around women, and them not being prepared to use AI as much as men. I found that very interesting insights. So maybe to conclude, shall we look at the future and shall we think about what the various scenarios may look like when it comes to responsible AI, the impact on society and the role that businesses have to play?

GDR: I would say that I'd see three different scenarios.

So the first scenario is business as usual. It's like what's happening mostly today is like organizations are trying to implement and adopt AI as fast as they can in order to increase productivity, to cut costs and to be more efficient. And that's what they are selling us. So responsible AI in that sense is more like a buzz word. The organizations claim to be using AI responsibly, but actually they're not doing much in that sense. And I think that this perspective and this scenario is going to be harder to sustain because there is more public awareness on AI risks. There is also more pressure from regulators and from consumers, from civil society. So at the end of the day, it's not going to be enough to say: "We are responsible. We are using AI responsibly". You'll need to demonstrate it in some sort of way.

And so that lead us to our second scenario, where I see there is this compliance-driven approach. So, in Europe, organizations will need to adapt the new regulations, the AI Act, and other places they are going to need to adapt to market. pressure and start putting frameworks into place: risks, assessments, documentation, governance frameworks, etc. So that's what's already happening and it's a necessary step, of course. But the risk there, as we said before the beginning of the interview, the risk there is that we see it just a tick boxing exercise. So we do enough. We are compliant and we stop there and it's kind of what we saw in the DE&I space before. You comply on paper, you do the things that you need to do at the right time. You do what's required, but fundamentally, you are not changing the way decisions are made. You are not changing your policies or your internal, functioning. So you actually don't do the work. And I think most organizations are going to be falling in this area. So it's like pure compliance.

And then there's a third scenario, which I believe is my dream scenario, and the most optimistic one. And this is there where there's real commitment, where organizations start seeing responsible AI not as a constraint, but as a source of value. So in terms of risk management, in terms of trust, in terms of competitive advantage, that they can use, just because it allows them to build better products that are more usable. So in that scenario, organizations are not just going to ask how do we use AI? Or, what can I do with this data that I have in my organization? But more meaningful questions like what kind of systems am I building? How are the systems going to affect individual society? How is AI going to transform my organization? Who is going to benefit from this transformation? Who is going to be left behind? Deeper questions and engage and involve the time and resources to meaningfully engage in this processes and to build AI as an integral part of your internal processes. Even if it takes more time, at the end, you'll see the results are going to be better. So this is where I see interdisciplinary teams come into place. More attention paid to societal impact, also things like participatory AI approaches in which you actually include the voices of affected communities, of different stakeholders in a meaningful manner within the way you develop and deploy AI. So, at the end of the day, I believe, and I really believe that organizations that navigate this well are going to be the ones that are actually involved in this framing. The ones that actually go through the questioning, through the processing of really embedding responsible AI practices into their workflows and the way of doing business. Because at the end of the day, what they are going to have is systems that are not just more inclusive, but systems that people can trust, and systems that will lessen the possibility of doing harm and then have a whole enormous PR problem in your company.

So, I think these are the three scenarios that I see and I hopefully will see more of the third scenario in the future.

int!: Yes, definitely. This is to the third scenario being the reality or the future of how businesses implement AI. I am absolutely with you on that one. I think it's really important, and to your point, I think there's enough public awareness related to the dangers. Not all of them. Not everyone understands all of the dangers that are related, but I think everyone knows that this has to be regulated in a way that is meaningful and there's a real business agenda to it. Because to your point, I think it can bring value to organisations. It can help them be more innovative. It can help them be more productive as well. But the framework that is there has to be thought through the lens of bringing everyone onto that journey for sure. Gabriela, thank you so much for your input and sharing your expertise and your experience in this space. It's been really, really interesting. and I look forward to hearing more from you and all the advancements in this space in the near future because it's really important and it's the latest revolution in this world. And we need to make sure it benefits humanity in general and humans, who are the users. And, to all of you watching this is Insight Connect, an initiative where cultural intelligence lands. For more topics around this subject, tune onto our website. There are plenty more interviews coming in the future. Thank you again, Gabriela. It was lovely to have you.

GDR: Thank you, Melanie. It was a pleasure.