Speaker 1: Welcome to tex Stuff, a production from I Heart Radio.
Speaker 1: Hey therein Welcome to tex Stuff. I'm your host, Jonathan Strickland.
Speaker 1: I'm an executive producer with I Heart Radio. And how
Speaker 1: the tech are you? I gotta treat view folks. Today
Speaker 1: I had the opportunity to speak with Bena Amnath, the
Speaker 1: executive director of Deloitte AI Institute. Vina is an accomplished
Speaker 1: technologist and expert in artificial intelligence. She's a coder, she's,
Speaker 1: you know, an engineer, and she's a great communicator too.
Speaker 1: She has appeared on numerous shows and panels talking about AI.
Speaker 1: She's also the author of a book called Trustworthy AI.
Speaker 1: And this was a fantastic opportunity to speak with someone
Speaker 1: who actually has a deep amount of experience in the
Speaker 1: field and really talk about some of the big concepts
Speaker 1: in AI and get a little more perspective on them.
Speaker 1: And I have to admit Beena's responses really open up
Speaker 1: the blinders that I have on And of course I'm
Speaker 1: like a lot of people, right I I go through
Speaker 1: life thinking I have a pretty good handle on this.
Speaker 1: I think I know what's going on, and then I
Speaker 1: meet someone else who has had a you know, a
Speaker 1: different experience, and especially a different depth of experience in
Speaker 1: a particular field and realize, oh gosh, I hadn't even
Speaker 1: considered some of these specific scenarios, for example. So I
Speaker 1: really very much enjoyed my conversation with Bena. She's also
Speaker 1: incredibly good at putting things in a way that are
Speaker 1: easily understandable. A lot of technologists, when you start to
Speaker 1: talk with them, they get really heavy with jargon or
Speaker 1: or concepts. That makes sense if you've had experience working
Speaker 1: in that area, but if you haven't, your eyes kind
Speaker 1: of glaze over and you just trust that what they
Speaker 1: say makes sense. That was not the issue with Bina.
Speaker 1: She is really good at talking about this stuff on
Speaker 1: a level that that the average person can easily understand,
Speaker 1: and yet also really stressing how AI is a very
Speaker 1: important component today. I mean, we're seeing it rolled out
Speaker 1: in all sorts of different ways across all different sectors.
Speaker 1: We mostly talked about business in this conversation, but clearly
Speaker 1: AI is everywhere. Whether we're talking about facial recognition technology
Speaker 1: that might be built directly into the camera on your phone,
Speaker 1: or maybe we're talking about a personal digital assistant, you know,
Speaker 1: something like the Amazon one. I won't say her name
Speaker 1: because some of you have her and she gets real
Speaker 1: like she she perks up when you say her name. Um,
Speaker 1: those sort of things obviously have components of AI built
Speaker 1: into them, but we were really looking at things like
Speaker 1: processes in business where you might need to use automation
Speaker 1: and artificial intelligence to make complicated processes more efficient and
Speaker 1: less human intensive. And so, yeah, this was a great
Speaker 1: conversation and I really feel like I learned a lot.
Speaker 1: I hope that you all enjoy it. And again, she
Speaker 1: does have a new book out. It's called Trustworthy AI,
Speaker 1: and there's a copy on the way to me, so
Speaker 1: I'm very eager to read it myself because just talking
Speaker 1: with Bina I felt like I was just scratching the surface.
Speaker 1: But you're gonna hear all that, So let's get to
Speaker 1: that interview. Bina. I want to welcome you to the show.
Speaker 1: I am so pleased to have an expert on AI.
Speaker 1: Trustworthy AI, no less, Welcome to tech stuff, Jonathan, thank
Speaker 1: you so much for having me on your show. I
Speaker 1: really enjoy your episodes, so I'm looking forward to having
Speaker 1: this conversation with you. I am as well. And one
Speaker 1: of the things that I like to do is kind
Speaker 1: of set some foundation for any kind of conversation around AI,
Speaker 1: because in my experience, and I'm sure you've experienced something
Speaker 1: similar chatting with people about AI, it seems like everyone
Speaker 1: has a different, sometimes very specific idea of what AI is.
Speaker 1: And I'm curious, how do you describe AI to people? Yeah,
Speaker 1: that's a great question to start with. So AI is
Speaker 1: a form of intelligence that uses machines to do things
Speaker 1: that traditionally required human intelligence. So it is artificial intelligence
Speaker 1: which is created artificially by machines. So now I like
Speaker 1: that description because it covers such a wide spectrum every
Speaker 1: thing from sort of the science fiction approach we've all
Speaker 1: seen about machines that seem to think like humans to
Speaker 1: a point where they usually become the threat. I mean,
Speaker 1: that's typically the way we look at it, which is
Speaker 1: I'm sure going to come into play when we talk
Speaker 1: about trustworthiness, because I'm sure a lot of people aren't
Speaker 1: aware how AI can sometimes be a danger, but not
Speaker 1: necessarily like sky Net from Terminator type danger. Let me
Speaker 1: elaborate on that description than a little bit more on
Speaker 1: you know that next level down on AI definition, Right,
Speaker 1: you know, the way I think about it. There are
Speaker 1: three types of AI. One is artificial narrow intelligence, which
Speaker 1: can do a very specific, narrow task that a human
Speaker 1: can do, like sort a bunch of photographs, Right, That's
Speaker 1: a very narrow specific task. So that's artificial narrow intelligence.
Speaker 1: And then there is a form of artificial general intelligence,
Speaker 1: which is a form of AI that can do any
Speaker 1: task that human beings can do, right, So it is
Speaker 1: pretty much everything that a human being can do. And
Speaker 1: then I think of a third category, which is artificial
Speaker 1: super intelligence, which is a form of intelligence which is
Speaker 1: smarter than all human beings combined and can do more
Speaker 1: things that human intelligence couldn't do. So when we talk
Speaker 1: about AI in the business world or in reality, it
Speaker 1: is where we are with AI. It's very much in
Speaker 1: that artificial narrow intelligence space. But when we hear a
Speaker 1: lot about the in the media or the hype and
Speaker 1: the fear, you know, it's talking really about that super
Speaker 1: intelligence phase, which is a form of AI that is
Speaker 1: smarter than all human beings combined and has more capabilities
Speaker 1: that human intelligence. And I think there's a big gap
Speaker 1: between reality and between where we you know, we are
Speaker 1: anticipating things to be right and and you know. Part
Speaker 1: of the reason is, you know, actual super intelligence is
Speaker 1: a lot of our human imagination, which is where AI
Speaker 1: was when I was studying years ago, right, So I
Speaker 1: do think there is value in imagination. I do think
Speaker 1: there is value in thinking of worst case scenarios so
Speaker 1: that you can address it. But the reality is we
Speaker 1: still today don't have the tools or the capabilities to
Speaker 1: build out artificial general intelligence or super intelligence, and we
Speaker 1: do not have that capability today. I see parallels in
Speaker 1: this as well, like I have a very similar description
Speaker 1: of autonomous cars, for example, like people talk about autonomous
Speaker 1: cars like we've reached level five autonomy, when really I
Speaker 1: would argue we're still around level two creeping into level three,
Speaker 1: but we we are not close to level four or five.
Speaker 1: And this is why I like having this kind of
Speaker 1: conversation right up front, so that people kind of set
Speaker 1: their expectations, because artificial intelligence can already do incredible things
Speaker 1: in these very narrow, narrow uses, and I'm blown away
Speaker 1: by it whenever I learned about that. But I do
Speaker 1: also see the the allure and sometimes the trap of
Speaker 1: extrapolating that beyond the narrow cases and thinking what happens
Speaker 1: when this goes beyond that, which it could very well happen,
Speaker 1: but we're not we're not at that stage yet. Um.
Speaker 1: But but I think of things like, yes, I think
Speaker 1: of things like like the image recognition like that to
Speaker 1: me is still an amazing thing to see developed, Like
Speaker 1: you know it is. Ever since I started covering tech,
Speaker 1: the ability has grown so fast. Like I remember when,
Speaker 1: at least on the consumer side, you might see something
Speaker 1: that was like detecting a ace, not recognizing a face,
Speaker 1: but detecting the structure that makes a face and for
Speaker 1: a camera, And now you know that that looks like
Speaker 1: stone age technology by comparison of what we're seeing today. Yes, Jarthan,
Speaker 1: And you I know you've been cowering tech for a
Speaker 1: long time. You know you've certainly seen seen the early evolution, right,
Speaker 1: but you know, and look, when I studied computer science,
Speaker 1: I did program assembly language programming basically using zeros and
Speaker 1: ones right that level. And you know the languages that
Speaker 1: I use, like Pascal and Fortran lots, you know those
Speaker 1: don't even exist today, right, So there's a whole evolution happening.
Speaker 1: And I do think that is a big component to
Speaker 1: imagine the future so that we can at least go
Speaker 1: towards take care of the risk, and focus on all
Speaker 1: the good things that you know AI and technology can do. Right,
Speaker 1: So imagination is a good thing, but not the fear
Speaker 1: a part. And also I think I think what you
Speaker 1: said kind of is a great message to anyone who's
Speaker 1: interested in really focusing on AI. The fact that you
Speaker 1: were working in assembly so close such a low level language,
Speaker 1: you get you get a real familiarity with what these
Speaker 1: machines can do and their their potential that I think, uh,
Speaker 1: you you almost lose when you start working on high
Speaker 1: level uh programming languages Like you you get so focused
Speaker 1: on what the programming language lets you do. But if
Speaker 1: you've worked out at that low level, you're like, hey, no,
Speaker 1: I know circuits and wires, Okay, I am, I am,
Speaker 1: I am one step away from this machine. Yeah, we
Speaker 1: we have to realize, you know, just like you know,
Speaker 1: when you talk about it today, will talk mostly about
Speaker 1: the software, but the hardware is also evolving. Right. We
Speaker 1: certainly don't have wax of those massive back from pure
Speaker 1: systems that we had to program, right. I think there
Speaker 1: is evolution happening in every dimension, and you know, it's
Speaker 1: it's part of the growth of AI or any technology
Speaker 1: if you think about it. M hmm, Well, I also
Speaker 1: want to know what you mean when you when you
Speaker 1: use the phrase trustworthy AI. So what is it that
Speaker 1: makes AI trustworthy? And what what's the what's the alternative?
Speaker 1: What is the untrustworthy side? Yeah, that's that's a great question,
Speaker 1: and that's something that you know. As a technologist, I'm
Speaker 1: enamored by all the cool things that AI can do
Speaker 1: because I just focus on all the value creation. But
Speaker 1: over the past a few years, as AI started becoming real,
Speaker 1: I also realized that there are you know, with all
Speaker 1: the good things that can do, there are negative consequences
Speaker 1: to it, right, and so I put that negative consequences
Speaker 1: in the under the bucket of untrusted untrustworthiness. Ethics is
Speaker 1: a big composed under of it, right, Whether the AI
Speaker 1: is fair or biased or transparent explainable, but also things
Speaker 1: like is it compliant with local regulations, does it have
Speaker 1: controls in place? Does it have governance in place to
Speaker 1: continuously monitor for it going wrongue because you know, Jonathan,
Speaker 1: today AI is mostly machine learning, so that it's learning
Speaker 1: and evolving. It's not that era when we developed code
Speaker 1: put it out there and the code states static and
Speaker 1: its behavior was very predictable. With AI, the outputs can
Speaker 1: change depending on inputs your feed and it's impossible to
Speaker 1: trade on all possible inputs. So trustworthy for me is
Speaker 1: you know, really, or is when you have addressed, when
Speaker 1: you have thought about and addressed all the possible negative
Speaker 1: things that this AI solution can cause. Well, I would
Speaker 1: love to kind of dive into a little bit more
Speaker 1: of that because one of the things that you said
Speaker 1: that really resonated with me was the idea of transparency,
Speaker 1: because I have covered this past episodes of tech stuff,
Speaker 1: but the sort of the black box problem of creating
Speaker 1: a system, for example, a machine learning system, and you
Speaker 1: have this this machine that's training itself over and over
Speaker 1: and over. Maybe it's adversarial training, maybe you actually have
Speaker 1: two systems that are set against each other and you're training,
Speaker 1: and the issues that can arise if you have distanced
Speaker 1: yourself so far from what the machine is doing that
Speaker 1: you are unable to determine the process by which it
Speaker 1: arrives at its conclusions. And that to me is one
Speaker 1: of those those pitfalls. Yes, but I would also challenge
Speaker 1: it a little bit, Jonathan, because that a whole synthesis
Speaker 1: of my book is that it depends on the use case.
Speaker 1: It is not a one size fit soul. So depending
Speaker 1: on where and to solve what problem are you using
Speaker 1: that DAIR solution, it is for that organization that teams
Speaker 1: decide if transparency is crucial. Right. If if your AI
Speaker 1: solution is being used to for patient care in a
Speaker 1: hospital system, then transparency is absolutely crucial, right. But if
Speaker 1: you are using the AI solution to predict when an
Speaker 1: X ray machine might fail, and you're able to predict
Speaker 1: at accuracy rate that this machine is going to fail
Speaker 1: in the next forty eight hours of call a technician,
Speaker 1: transparency may not be as crucial, right. So I think
Speaker 1: transparency is crucial depending on the use case. And that's
Speaker 1: true for all the other dimensions as well, even fairness
Speaker 1: and bias, which we hear a lot about. So it
Speaker 1: really depends on the use case that that you're using
Speaker 1: the I fall Hey there, Jonathan back at the home studio,
Speaker 1: just here to say we are going to take a
Speaker 1: quick break, but we'll be back with more with Bena Amanath,
Speaker 1: the executive director of Deloitte AI Institute. Bias doesn't necessarily
Speaker 1: mean negative, depending upon the use case of the technology.
Speaker 1: In some cases you need to have a biased system
Speaker 1: because it's specifically meant to be weighted to do one
Speaker 1: thing versus another, and without the bias it doesn't do that.
Speaker 1: But the way we typically hear about bias is when
Speaker 1: it is making a negative impact, when it's something for
Speaker 1: it's like like the facial recognition technologies. We've heard plenty
Speaker 1: about that. So it is interesting to me, And uh,
Speaker 1: I'm curious, like, what are what are some of the
Speaker 1: uses of AI you're seeing in technology now that you
Speaker 1: find really exciting. Yeah, no, I think you know, we're
Speaker 1: still very early on in this technology evolution and there
Speaker 1: are still so many use cases to be solved, so
Speaker 1: many industries to take a I too right to point
Speaker 1: about bias and its relevance. I completely agree with you that,
Speaker 1: you know, it depends on the use case, and it
Speaker 1: goes back to that first question we talked about, right,
Speaker 1: how AI is really emulating human intelligence, which means that
Speaker 1: it is going to carry over the biases of the
Speaker 1: humans that are building it. Right, But as as a
Speaker 1: business or as an organization who's looking to use an
Speaker 1: air solution, who's looking to develop an a solution, they
Speaker 1: really have to, you know, bring together the stakeholders to
Speaker 1: discuss and decide how crucial is fairness or unbiased ness
Speaker 1: important in this particular AI use case. And easy one
Speaker 1: out is if it doesn't involve human data, then you
Speaker 1: probably don't have to worry as biased as a factor
Speaker 1: and address it. And if it does involve human data,
Speaker 1: then again there is weightage in what right if there
Speaker 1: is biased at in an algorithm that is providing personalized marketing,
Speaker 1: that you know that there is a weight to it.
Speaker 1: And if it is if there is biased in an
Speaker 1: algorithm that is supporting law enforcement decisions, that's a higher rate, right.
Speaker 1: And it's really about rating it, you know, weighing it
Speaker 1: and deciding which ones are the one where biases acceptable
Speaker 1: and you can still proceed and get value from the
Speaker 1: AI solution, and which are the ones where it is
Speaker 1: absolutely not acceptable and you need to stop and figure
Speaker 1: out and alternate way to solve for that problem. It's
Speaker 1: fascinating because it to me this is starting to sound
Speaker 1: and I agree with you, like the machines we build
Speaker 1: are in large part reflections upon ourselves, especially when we're
Speaker 1: talking about coding and software. I mean obviously that's going
Speaker 1: that's a creative process. I don't know that everybody views
Speaker 1: it that way, but I think of it very similar
Speaker 1: to creating any kind of creative work. It's a reflection
Speaker 1: of your process and your you know, the things that
Speaker 1: are important to you, the things you've prioritized. And it
Speaker 1: makes me think of how we're in an era now
Speaker 1: where I'm getting a little in the weeds here, but
Speaker 1: we're in an era where we're more likely to address
Speaker 1: things like, uh, mental health and the fact that we
Speaker 1: need to be mindful and we need to improve ourselves.
Speaker 1: And it's almost like taking that same approach, but applying
Speaker 1: that sort of thinking to designing a system so that
Speaker 1: we are being mindful to create the best system for
Speaker 1: whatever purpose it is it's intended to address. Yeah, you've
Speaker 1: got it exactly right. The way I think about it is,
Speaker 1: how can we reduce the unintended consequences? Right? We know
Speaker 1: there are going to be risk associated with it, How
Speaker 1: are we going to have a discussion prior to putting
Speaker 1: the solution out into the world and then you know,
Speaker 1: see all the negative impacts. Can we have a proactive
Speaker 1: discussion as part of your project planning meeting or your
Speaker 1: design meeting right to proactively identify what are the ways
Speaker 1: this could go wrong and fix it. Johnathan, you know,
Speaker 1: the easiest example that I can give is we're living
Speaker 1: in this very interesting era where you know, AI as
Speaker 1: a core technology is developing and you know, there are
Speaker 1: all these the value that you're getting from it, and
Speaker 1: then there are all these negative things that can happen.
Speaker 1: So think about you know, way back when when you
Speaker 1: know the cars were first invented, right, we didn't even
Speaker 1: have proper roads. We didn't have seed belts, we didn't
Speaker 1: have speed limits, right, and be in that phase where
Speaker 1: there are cars running on the road. They're taking us
Speaker 1: from point to point be faster, so we want to
Speaker 1: use it, but we don't have the seat belts put
Speaker 1: in place, we don't have the speed limits set in place,
Speaker 1: so you're going to see accidents. But we are humans.
Speaker 1: We're going to learn from it and we're going to
Speaker 1: come up with those speed limits. We're going to figure
Speaker 1: out what are those card rails, and it is going
Speaker 1: you know, we are going to you know, achieve a
Speaker 1: point where you know, we have those guard rails in
Speaker 1: place so that you can run with AI faster. It's
Speaker 1: just that this interim phase is when you know, we
Speaker 1: have to figure it out out in tandem while it's
Speaker 1: running in the real world, causing accidents. And in some
Speaker 1: cases that's that those accidents can be things where you
Speaker 1: have it maybe in a test environment and you think, oh,
Speaker 1: this isn't behaving the way I thought it was. But
Speaker 1: you know, thank goodness, it hasn't been deployed out in
Speaker 1: the real world for or within your company's UH processes,
Speaker 1: so you think, oh, well, it didn't wipe out all
Speaker 1: of our revenue because it's in a test environment. UH.
Speaker 1: And in other cases, I see I see some companies.
Speaker 1: I'm not gonna name names, Bina, I'm not gonna put
Speaker 1: anyone on blast here, but I have seen some companies
Speaker 1: that have taken that kind of idea and applied it
Speaker 1: in UH specific deployments of technology where there can have
Speaker 1: some some real world negative consequences to end users. UM.
Speaker 1: And that to me has always a concern I find, yeah,
Speaker 1: I find that I find it hits me wrong. Yes,
Speaker 1: And that's the reality of how we've evolved as a
Speaker 1: technology in their technology space. It's a bunch of you know,
Speaker 1: technologists coming together and building these cool, new shiny technologists. Look.
Speaker 1: You know, as I said, I am a technologist in
Speaker 1: my DNA my training, and it's very easy to just
Speaker 1: focus on all the good things that can do. But
Speaker 1: with AI, now that realization has hit, you need other
Speaker 1: you know, skill sets at the table, whether it is
Speaker 1: a social site, is philosophers, legal and compliance to help
Speaker 1: us figure out those seedbells and the you know, the
Speaker 1: speed the lanes, you know, because technologies by themselves cannot
Speaker 1: do it. So you'll see more of the discussions coming
Speaker 1: around ethics and which is resulting in new roles and
Speaker 1: new jobs, which becomes core and part of your engineering process. Right,
Speaker 1: So that scope of who is involved in designing and
Speaker 1: developing AI is definitely increasing. And the other big part,
Speaker 1: you know, and this has been a challenge since I
Speaker 1: started in tech. You know, there's a lack of diversity
Speaker 1: in tech. It's a reality, right, But unfortunately, because AI
Speaker 1: is so closely tied to human intelligence, if you don't
Speaker 1: have enough diversity from you know, not only from a gender,
Speaker 1: race at necessity perspective, but even a diversity of thought, right,
Speaker 1: you're the AI solution you built is not going to
Speaker 1: be as robust as it could be if you had
Speaker 1: a diverse team at the table. Right, you've probably heard
Speaker 1: of that classic example of you know, the robotic vacuums, right,
Speaker 1: how it was designed and now it was built out.
Speaker 1: And then in the Eastern cultures it's normal to sleep
Speaker 1: on the floor and it sucked up human somebody who's
Speaker 1: sleeping their hair because it was never trade on it.
Speaker 1: It didn't come you know, it didn't come to the discussion,
Speaker 1: and it was being designed because nobody was there from
Speaker 1: that culture. Right. So I think, you know, the realization
Speaker 1: that you need more diversity at the table, you need
Speaker 1: more controls in place. It's all coming to the forefront.
Speaker 1: I definitely see companies addressing it. But the the DNA
Speaker 1: will now has been oh, look at all the cool
Speaker 1: things this technology can do, let's go put it out right.
Speaker 1: But I do think, you know, companies are getting mindful
Speaker 1: about it and hopefully we'll reduce the number of unintended consequences. Yeah.
Speaker 1: I see the same thing reflected in the open source community,
Speaker 1: where you have an open source approach to developing software,
Speaker 1: and because it's open and and anyone interested and capable
Speaker 1: can contribute ideas get tested, very quickly. New new perspectives
Speaker 1: get incorporated very quickly. Things that are working stick around,
Speaker 1: things that don't work get improved. And the way I've
Speaker 1: described it to other people is, if you have a
Speaker 1: closed off garden that you're working on, you're only as
Speaker 1: good as the smart people who happen to work for you.
Speaker 1: And if you go with this other approach where you
Speaker 1: purposefully open it up, which is like the biggest version
Speaker 1: of let's let's try and get as much diversity of
Speaker 1: thought in here as possible. Uh, you don't have that
Speaker 1: limitation because you've You've just said, well, now the world
Speaker 1: is I mean it's not the whole world, but but
Speaker 1: effectively the world. The world can contribute if if they,
Speaker 1: if they wish, and uh agree, I think having that
Speaker 1: diversity is absolutely key to creating solutions that work for
Speaker 1: as many people and as many potential uses of that
Speaker 1: technology as possible. I being a a a white man
Speaker 1: in the United States, I am I am essentially the
Speaker 1: catered to audience for a lot of tech, and so
Speaker 1: I've seen how things that were made to work really
Speaker 1: well for me do not work for some other people.
Speaker 1: And that's such a tiny little microcosm when we're looking
Speaker 1: at you know, the GREA and scope of tech which
Speaker 1: goes so far beyond just consumer electronics. UM I absolutely
Speaker 1: agree that that diversity is is required if we're going
Speaker 1: to have a i that is truly trustworthy. Yeah, exactly,
Speaker 1: And you know, and then but there it's not never.
Speaker 1: It's never as straightforward as we tend to simplify it
Speaker 1: down to, right, like when we talk about explainability. There
Speaker 1: there are real challenges and those are real business challenges
Speaker 1: on even when you go down the open source route right,
Speaker 1: a lot of time, if you go too much on
Speaker 1: the explainableitypath, you know, you you have to still share
Speaker 1: data and algorithms and those are strategic assets and it
Speaker 1: can result in compromising your company's i P. Right, it
Speaker 1: can result in you know, security hacks because the more
Speaker 1: explainable you make it, it is more susceptible to manipulation
Speaker 1: if it's functionality is fully understood, the privacy aspect of it,
Speaker 1: prioritizing playability and you know, how do you make sure
Speaker 1: you hit a balance of why you are making sure
Speaker 1: you're mitigating the risk but at the same time protect
Speaker 1: your organizational i P. That's that's a that's a solution.
Speaker 1: That's that there is no one single answer. It is
Speaker 1: for the stakeholders to come together and discuss it and
Speaker 1: identify were that balances, because it's going to be different
Speaker 1: depending on your business. It seems to me like you're
Speaker 1: saying the real world is a complicated place and there's
Speaker 1: a lot of different shades of complexity to it, and
Speaker 1: that I can't just simply uh summarize it in a
Speaker 1: black and white approach, which I greatly appreciate, uh, and
Speaker 1: that that's interesting to me too. I'm glad to have
Speaker 1: that perspective because again, like as a as a communicator
Speaker 1: for tech, uh, I know that I too fall into
Speaker 1: the same sort of pitfalls of oversimplifying for the purposes
Speaker 1: of trying to get a concept across, because to really
Speaker 1: dive into it, you start to you start to feel
Speaker 1: like they're there are so many threads that you can't
Speaker 1: see the rope and that or you can't see the
Speaker 1: forest for the trees if you prefer. But but that's
Speaker 1: that's very important to remember, and I think it is
Speaker 1: a great reminder that again, like we said at the top,
Speaker 1: that the use for this technology kind of defines the
Speaker 1: approach that you need to take in order to make
Speaker 1: certain that you're you're getting the result that you want.
Speaker 1: UM from a really high level, can you kind of
Speaker 1: talk about your concept of what what it is? This
Speaker 1: is almost a trick question because there's so many different variations,
Speaker 1: but what what what an organization's process would be when
Speaker 1: considering to implement AI solutions like high high level approach. Yes,
Speaker 1: Historically it's always been you know, how can we use
Speaker 1: ARE to solve this business problem? And what's the r
Speaker 1: O I what you know? How much profits are we
Speaker 1: going to increase by doing this? Or how much costs
Speaker 1: are we going to save by doing this? Trust me,
Speaker 1: I've done this project and you know that's how you
Speaker 1: know every conversation starts because we want to make use
Speaker 1: technology to drive more business value, right, whether it is
Speaker 1: through customer engagement, optimizing our existing process and so on.
Speaker 1: I think the discussion that that if you are serious
Speaker 1: about getting making your AI trustworthy, the discussion that needs
Speaker 1: to happen upfront is defining what does trustworthy I mean
Speaker 1: for for my organization? Right? And uh and it could
Speaker 1: be different depending on the organization, It could be different
Speaker 1: depending on the use case. But having those high level principles,
Speaker 1: and there are plenty of principles out there, there are
Speaker 1: plenty of frameworks out there, but I think every organization
Speaker 1: needs to think about what are the key pillars that
Speaker 1: they agree upon and that they would never want to
Speaker 1: void it right. And once you have those, then next
Speaker 1: step is to decide to make sure every employee within
Speaker 1: your organization understands it. Because it's not just your I
Speaker 1: T team, It's not just the engineers of the data
Speaker 1: scientists who need to understand ethics. It's that marketing marketing
Speaker 1: account person who is looking at using an AI solution,
Speaker 1: buying it from a vendor to use it within your company.
Speaker 1: They need to make sure that they are asking the
Speaker 1: questions which ensure trustworthiness and do they do is the
Speaker 1: software they're buying, has it been tested for fairness? What
Speaker 1: was it tested for? So every employee within the organization
Speaker 1: needs to understand what distrustworthy I mean for my company
Speaker 1: and how do I how do I make it, how
Speaker 1: do I use it in my role? So role specific training.
Speaker 1: And then the other crucial factor to decide, and we've
Speaker 1: see variations of it in the industry, is you know
Speaker 1: whether it is getting a cheap AI Ethics officer or
Speaker 1: setting up an AI thinks advisory board right, making sure
Speaker 1: that there is somebody who is responsible to keep you know,
Speaker 1: to keep this moing within the organization is super important.
Speaker 1: That's more from a people perspective. And then the last
Speaker 1: thing is really looking at your existing processes. I don't
Speaker 1: think you need to completely come up with new processes
Speaker 1: or new controls, but just adding in an trustworthy check
Speaker 1: in your existing engineering processes or in your existing development
Speaker 1: process or your procurement process to make sure you're checking
Speaker 1: for the trustworthiness of any AI that tool that you
Speaker 1: buy or that you build, you know, having in addition
Speaker 1: to the r O, I ask Sen spent ten percent
Speaker 1: of your time to brainstorm on what are the ways
Speaker 1: this could go wrong? Right? And capture it and when
Speaker 1: you build that technology, put those guard rails in place.
Speaker 1: Now it is guaranteed you It is impossible to identify
Speaker 1: all the possible ways it could go wrong, but even
Speaker 1: if you get you know the ways it could go wrong,
Speaker 1: it is better than not thinking about it and not
Speaker 1: addressing it. So that is a very comprehensive way you
Speaker 1: can do it. But it is all easy. It fits
Speaker 1: in with the existing trainings and processes that you already
Speaker 1: have in your business. Right. I gotta say, like as
Speaker 1: as someone who is a technologist and uh and coming
Speaker 1: at this from that angle, that was such a human
Speaker 1: centric kind of answer. I really appreciate that. I've had
Speaker 1: a lot of discussions with various leadership around different companies
Speaker 1: and this idea of of having that explanation and getting
Speaker 1: buy in from different departments so that everyone's on the
Speaker 1: same page and they have an understanding of the purpose
Speaker 1: of a tool, how it's going to be implemented, what
Speaker 1: we expect to get out of it. Uh. That's actually
Speaker 1: crucial for anything, whether it's a I or not. But
Speaker 1: because I've seen so many examples of companies where you
Speaker 1: have one department who's like a business development team wanted
Speaker 1: us to put this in and I don't understand why.
Speaker 1: And if they don't understand why, then you don't get
Speaker 1: as good output on the other end of it. I
Speaker 1: think making that part of the conversation just as much
Speaker 1: as you know, determining the approach to get a trustworthy AI,
Speaker 1: I think that's absolutely crucial. Yes, And you know, a
Speaker 1: lot of times we think it's a technology problem to fix,
Speaker 1: right it's it's a technology. You know, to build trustworthy
Speaker 1: air you need to you know, it's a technology problem.
Speaker 1: It's your data scientists and engineers, which you think about it,
Speaker 1: but that's that's not the case, right, It's a it's
Speaker 1: the entire group that needs to come together. And the
Speaker 1: risk is not just from a technology perspective. It's a
Speaker 1: brand and reputation rusk. There's financial consequences, there's customer satisfaction consequences,
Speaker 1: there is so many other risks associated with if your
Speaker 1: AI is not trustworthy. Bina and I have a little
Speaker 1: bit more to talk about with AI, but before we
Speaker 1: get to that, let's take another quick break. I remember
Speaker 1: covering that over in the European Union there were various
Speaker 1: departments that were even talking about concepts that again are
Speaker 1: let science fiction far off concept, but even the concept
Speaker 1: of of granting personhood toward sufficiently advanced AI for the
Speaker 1: purposes of figuring out accountability and responsibility for when something
Speaker 1: goes wrong, who gets held accountable when the AI doesn't
Speaker 1: work right? What's your take on that. I think, you know,
Speaker 1: we might reach at that at some point, but in
Speaker 1: the interim till we don't have that kind of you know,
Speaker 1: rules or laws. I think it's absolutely you know, one
Speaker 1: of the components dimensions of trustworthy A is defining accountability upfront,
Speaker 1: meaning if the AI goes wrong, who is accountable for it?
Speaker 1: Who's going to phase the Senate hearing? Who's going to
Speaker 1: pay the fine? Is it the data scientists to milit it,
Speaker 1: is it the c I O who approved the project?
Speaker 1: Is it the CEO or is it a board member? Right? So,
Speaker 1: and the good news with that one, you know, talking
Speaker 1: about accountability upfront makes everybody proactively think about for the
Speaker 1: ways it could go wrong, because you don't want to
Speaker 1: put your name on something that might go wrong and
Speaker 1: you have not thought about it. So until we get
Speaker 1: to that, you know, machine citizens citizen rights level, I
Speaker 1: think you know, even today there is a dimension of
Speaker 1: trustworthiness which is really around defining putting in a name
Speaker 1: for who is accountable when your AI goes wrong. I
Speaker 1: agree that that's important. I have seen some of those
Speaker 1: Senate hearings with various UH tech people sitting in the sea,
Speaker 1: and I know that if I were in one of
Speaker 1: these conversations, I would not want to be that person.
Speaker 1: And making sure we specifically define who that person is
Speaker 1: and that it's not me would be top of my
Speaker 1: priority life. Well, I'm also curious then. Uh. So we've
Speaker 1: seen in a similar sense some movement on things like
Speaker 1: autonomous cars. Uh. In a similar note, I'll talking about accountability,
Speaker 1: where we're starting to see more governments try and consider
Speaker 1: who is accountable for any accidents that might have happened
Speaker 1: under cars autonomous or semi autonomous operation. Obviously that's been
Speaker 1: a big point of discussion here in the United States,
Speaker 1: and uh, this is one of those things. How how
Speaker 1: how closely tied do you think do technology experts need
Speaker 1: to be with say politicians who may not have the
Speaker 1: insight into tech, but yet are also responsible for creating
Speaker 1: and enacting policy that's going to have an effect on tech.
Speaker 1: Is do you see there being more cross talk? Yeah,
Speaker 1: you know, unlike the car example and the seed belt
Speaker 1: and speed limit example. You know, AI does need an
Speaker 1: understanding of technology so to come up with those speed limits.
Speaker 1: It is so, you know, and we've honestly entered that
Speaker 1: era where collaboration is king, right. We have to make
Speaker 1: sure that regulators and technologies, uh, policymakers, they have collaborating
Speaker 1: and each one is learning from the other. To come
Speaker 1: up with the best possible guard rails or regulations or laws,
Speaker 1: because this is not something that can be done in isolation,
Speaker 1: and like that auto speed limit example. So I think
Speaker 1: we're going to see more whether it is an entities
Speaker 1: being set up who will drive this collaboration, but there
Speaker 1: is definitely, you know, across the globe technologists being pulled together,
Speaker 1: whether as an advisory committee or a council. That is
Speaker 1: happening now, and you know, I do think we will
Speaker 1: start seeing results of that collaboration coming out sooner rather
Speaker 1: than later. I think I also believe that just like
Speaker 1: I was talking about every organization should train all their employees,
Speaker 1: I think every everybody who is involved in the regulation
Speaker 1: making process should have a basic understanding of AI, level
Speaker 1: of AI fluency, or you know, an understanding of what
Speaker 1: does machine learning really mean, what can it do, what
Speaker 1: can it not do? So I call it the AI
Speaker 1: literacy training, right, So I think it's that's like ground
Speaker 1: stakes to drive a productive collaboration. But I think this
Speaker 1: is the time for people like you and me, Jonathan,
Speaker 1: to really step up and make sure that we're collaborating
Speaker 1: closely so that that it's informed and informed and relevant
Speaker 1: regulation or relevant policy that's put together. I think relevance
Speaker 1: is is absolutely the right word to use. UH. Again,
Speaker 1: I'm not putting anyone on blast, but there have been
Speaker 1: plenty of stories of people, whether they are in the
Speaker 1: regulatory field or general politics, where their level of tech
Speaker 1: savvy is probably not even measurable based upon some of
Speaker 1: the things we've seen, and that is that is terrifying
Speaker 1: when you realize the reach and the effect of technology
Speaker 1: and how if you have a misunderstanding of it, you
Speaker 1: can tackle something that's not really a problem, but you've
Speaker 1: built it up as if it were while completely missing
Speaker 1: things that we absolutely need to pay closer attention to.
Speaker 1: So I I do try to to make literacy one
Speaker 1: of those things that I push for and hopefully I
Speaker 1: succeed more often than I fail. Yeah, it's we live
Speaker 1: in this era now that you know, at least in
Speaker 1: the UH. In the corporate world, right, we're seeing more
Speaker 1: and more boards getting more technology savvy. Leaders are leaders
Speaker 1: who understand technology so that because every company uses technology,
Speaker 1: uses AI no matter which industry they're in, right, So
Speaker 1: we're seeing that composition of boards changing, right, And I
Speaker 1: don't think we're very far from the time when you know,
Speaker 1: having a basic AI or technology understanding will be almost
Speaker 1: a prerequisite. Right Again, as I said, we're living in
Speaker 1: this interim crazy phase where there's a lot of things
Speaker 1: happening and we don't necessarily have all the foundations set up.
Speaker 1: The exciting news is for our generation, Jonathan, this is
Speaker 1: our opportunity. Right the work we do today is going
Speaker 1: to be setting the foundation for future generations. So I think, uh,
Speaker 1: you know, having that basic AI literacy, No, it's not
Speaker 1: set up, but you know, we we now understand that,
Speaker 1: you know, everybody who is involved in policymaking our regulations
Speaker 1: need to have that basic understanding. So let's make sure
Speaker 1: that you know they have that. That's great, it's it's
Speaker 1: it's looking at something that I have defined as a
Speaker 1: problem and you have defined as an opportunity, which I
Speaker 1: needed to hear honestly, because that's the kind of optimism
Speaker 1: that I find really motivating. Been a thank you so
Speaker 1: much for being on the show. Your book Trustworthy AI.
Speaker 1: I have a copy coming to me. I have not
Speaker 1: yet been able to read it. I am so eager
Speaker 1: to go cover to cover on this because just this
Speaker 1: this conversation has really energized me, and um, you know,
Speaker 1: when you have a podcast about tech and you've done
Speaker 1: more than sevent episodes, sometimes you feel like I've said
Speaker 1: everything there is to say about that, and then I
Speaker 1: have a conversation like this and I realized, this is
Speaker 1: an Iceberg situation and I've just touched the very tip
Speaker 1: of it. There an entire world beneath the surface of
Speaker 1: the water that I haven't even scratched. So thank you
Speaker 1: so much for coming onto the show, Jonathan. This is
Speaker 1: a very energizing conversation for me as well. Thank you
Speaker 1: so much for having me on your show. Once again,
Speaker 1: I have to thank Bena Amanath for coming on the show. Uh.
Speaker 1: I was thrilled at this opportunity when I first got
Speaker 1: the email suggesting that I have her on my show,
Speaker 1: because to be totally clear, her team reached out to
Speaker 1: me and I just didn't even think about that possibility.
Speaker 1: I am so glad that I followed up with that.
Speaker 1: I do plan on having more interviews on this show
Speaker 1: in the near future. I've got a couple more lined up.
Speaker 1: I'm gonna try and do that more frequently. It is
Speaker 1: I'm gonna be transparent with all of you. It is
Speaker 1: very tricky for me because scheduling UH is tricky. People
Speaker 1: are very busy, and it gives me a lot of
Speaker 1: anxiety just being absolutely transparent with all of you out there.
Speaker 1: The the the process of scheduling gives me a lot
Speaker 1: of anxiety. So it's something I'm working through and I'm
Speaker 1: trying to get more people on the show one because
Speaker 1: there's so many interesting people out there. And just with
Speaker 1: this conversation with Bena, I really got that that feeling
Speaker 1: of I need this because it is giving me more
Speaker 1: perspective than what I have and I'm I don't want
Speaker 1: tech stuff to just be a narrow laser focus of
Speaker 1: what Jonathan thinks about tech. Secondly, UM, you know, I
Speaker 1: think that it benefits the show obviously to have that
Speaker 1: that extra voice in there, and that means that it
Speaker 1: becomes more enjoyable because despite my enormous ego, I realize
Speaker 1: I cannot be the most entertaining person in all the world, UH,
Speaker 1: no matter how hard I try. So I hope you
Speaker 1: all enjoyed this. If you have suggestions for future topics,
Speaker 1: maybe you have suggestions for future guests I should try
Speaker 1: and get on the show. Reach out to me. Uh,
Speaker 1: I promise I will do my best to get that
Speaker 1: person on the show. I can't promise that it will happen,
Speaker 1: but I'll try and I'll work through this weird stress
Speaker 1: I get whenever it comes down to trying to schedule
Speaker 1: things and uh, and just to be clear, Bena was
Speaker 1: amazing because we actually tried to record that interview on
Speaker 1: one day but had a technical issue ended up having
Speaker 1: to reschedule. She was amazing. It was really good about
Speaker 1: all that. So despite all of my anxiety, everything went great,
Speaker 1: which I think is this isn't meant to be a
Speaker 1: therapy session. But I think that's very typical for me,
Speaker 1: where I get worked up about something turns out that
Speaker 1: something wasn't really that big a deal. It was just
Speaker 1: the anticipation of it that was the problem. So if
Speaker 1: any of you out there suffer from something like that,
Speaker 1: you know you have that same sort of experience. Listen,
Speaker 1: I got your back. I know how it is. It
Speaker 1: is frustrating, but you can do it all right. Eight
Speaker 1: PEP talk Over, Episode over. I hope you enjoyed it.
Speaker 1: I am on vacation for the rest of the week,
Speaker 1: so you should expect some classic episodes for the rest
Speaker 1: of this week. But that doesn't mean they're bad. It
Speaker 1: just means they're old, just like me. I'm old, but
Speaker 1: I'm not bad, and I will talk to you again.
Speaker 1: Oh if you want to reach out to me, you
Speaker 1: gotta do it on Twitter. The handle for the show
Speaker 1: is tech Stuff h s W There. Now I get
Speaker 1: to say the end catchphrase, I'll talk to you again
Speaker 1: really soon. Tech Stuff is an I Heart Radio production.
Speaker 1: For more podcasts from my Heart Radio, visit the i
Speaker 1: Heart Radio app, Apple Podcasts, or wherever you listen to
Speaker 1: your favorite shows.