Speaker 1: Bloomberg Audio Studios, Podcasts, radio News. Hello and welcome to
Speaker 1: another episode of the Odd Blots podcast. I'm Tracy Alloway.
Speaker 2: And I'm Joe Wisenthal.
Speaker 1: Joe, what's been your favorite chat GPT or claude prompt
Speaker 1: so far?
Speaker 3: You know, it's funny because I have a lot of
Speaker 3: fun with them, and also I use them for serious things.
Speaker 3: So I'll like upload conference call transcripts and say, tell
Speaker 3: me what this company said about labor market indicators or
Speaker 3: something like that, and that'll be extremely useful for that.
Speaker 1: Wait, do you actually find that more efficient than just
Speaker 1: doing a word search for like labor or working? I don't.
Speaker 1: I hate uploading stuff because you can only do it
Speaker 1: in like fragments.
Speaker 3: No, what, Tracy, Oh, let me, I'll show you how prompt? Okay, No,
Speaker 3: I get a lot of professional use out of the
Speaker 3: various AI tools, but I also, you know, have a
Speaker 3: lot of fun with them. And there's even a song
Speaker 3: and I'm not going to say which one that I wrote.
Speaker 3: I didn't use the lyrics. No, I did not like
Speaker 3: because it's very good. Wait what did you use?
Speaker 1: Did he give you an actual melody? What happened?
Speaker 4: No?
Speaker 3: So there was a song that I liked, okay, and
Speaker 3: the song title sort of rested upon a pun okay,
Speaker 3: and so I asked chat GPT to come up with
Speaker 3: another song that sort of like had a similar twist
Speaker 3: based on the headline of that song. I needed basically
Speaker 3: a song prompt idea.
Speaker 1: This opens up a whole can of worms. No, this
Speaker 1: is actually the perfect segue into what we're going to
Speaker 1: talk about today, because for you and I, using something
Speaker 1: like a chat GPT, we don't really have the same
Speaker 1: concerns that a proper company or large which corporation would have, Like,
Speaker 1: it doesn't really matter to us if the answer is wrong.
Speaker 1: I mean, ideally you would like it to be correct,
Speaker 1: but if I'm just asking some silly question, it doesn't
Speaker 1: really matter what chat gpt spits out at me. And
Speaker 1: also copyright kind of doesn't matter, so we don't care
Speaker 1: what it spits out in terms of who owns it,
Speaker 1: and also we don't care what we're putting in in
Speaker 1: terms of who owns that. That's right, But if you
Speaker 1: are a company you are thinking about generative AI very differently.
Speaker 2: I just want to say one thing, which is.
Speaker 1: That your defense Okay, defend yourself.
Speaker 3: No, No, I'm not even trying to defend myself. If I upload, say,
Speaker 3: you know, the McDonald's earning transcript, and I say, what
Speaker 3: does McDonald say about the labor market, then there's some quote.
Speaker 3: I always go back and check that that quote is
Speaker 3: actually in there. So I do very good, you know,
Speaker 3: I'm not just blindly relying on it. I do also
Speaker 3: do my own work and everything. But yeah, it's very true.
Speaker 3: Like so I can say I get a tremendous amount
Speaker 3: of use from chat, GPT or Claude or whatever, and
Speaker 3: it is very useful to me. But it makes mistakes sometimes,
Speaker 3: and if you think about deploying AI in the sort
Speaker 3: of enterprise world, then maybe like a one percent mistake
Speaker 3: raid or a one percent hallucination or you ever want
Speaker 3: to call them, is just completely unacceptable and a level
Speaker 3: of risk that makes it almost unusual for professional purposes.
Speaker 1: Absolutely. And of course the other thing with AI is
Speaker 1: there is still this ongoing, very heated debate about how
Speaker 1: transformational it's actually going to be. So you and I
Speaker 1: are using it as you know, a productivity hack in
Speaker 1: some cases, or maybe to generate song lyrics or even
Speaker 1: songs in some cases, but what is the true use
Speaker 1: case for this particular technology. There's still a lot of
Speaker 1: debate about that, and so I'm very pleased to say
Speaker 1: we do, in fact have the perfect guest. We're going
Speaker 1: to be speaking to someone who is implementing AI at
Speaker 1: a very, very large financial institution. We're going to be
Speaker 1: speaking with Marco Urgenti, the chief information officer at Goldman Sachs. Marco,
Speaker 1: thank you so much for coming on of thoughts.
Speaker 4: Thank you for having me.
Speaker 1: Marco tell us what a chief information officer does at
Speaker 1: Goldman Sachs. Whenever I see CIO, I always think chief
Speaker 1: investment officer, as it's very confusing. Yeah, so what does
Speaker 1: the other CIO do?
Speaker 4: So last week I was in Italy visiting my mother.
Speaker 4: She's eighty three, and she obviously doesn't know much about
Speaker 4: technology or banking, and so she said, what do you
Speaker 4: do with Coleman? And I said, you know, I just
Speaker 4: tried to simplify. I say, make sure that the printers
Speaker 4: don't run out of And interestingly, the CIO job has
Speaker 4: been traditionally associated with the word it.
Speaker 2: Okay and it.
Speaker 4: I tell you, talk to any technologist, they don't want
Speaker 4: to be classified as IT.
Speaker 3: Right, because those are you associated with those are the
Speaker 3: people who like, see if the ethernet cable with.
Speaker 4: Those are the ones who tell you that those that
Speaker 4: you know, I mean, I have a lot of respect
Speaker 4: for it, but generally you go to the IT department
Speaker 4: when something doesn't work, okay, And so it's very back
Speaker 4: office and something that attracted me to this job. I've
Speaker 4: been here for five years and this is the first
Speaker 4: time that I do like a CIO job. Before I
Speaker 4: was doing more like, you know, creating technology, et cetera,
Speaker 4: and service. I can talk about that, but is the
Speaker 4: fact that the role of a CEO has actually changed
Speaker 4: quite a bit, and now it's about really asking the question,
Speaker 4: you know, how do we implement technology in order to
Speaker 4: achieve our strategic objectives and actually to be differentiated, And
Speaker 4: it's really sitting at the strategic table of the firm.
Speaker 2: Okay.
Speaker 4: So today we live in a world where obviously a
Speaker 4: lot of the things that we want to do, or
Speaker 4: every company wants to do, are really kind of determined
Speaker 4: by how good you are at technology. And so I
Speaker 4: think the role of the CIO has changed quite a bit.
Speaker 4: And now, you know, I would define it as in general,
Speaker 4: defining the technology strategy of a firm and also making
Speaker 4: sure that you have the right culture in the engineering
Speaker 4: team in order to execute on that.
Speaker 3: What's the day to day look like? Like, what's the
Speaker 3: typical day you get into the office and then what.
Speaker 2: Do you do?
Speaker 4: Well? I mean I get into the office, and I generally,
Speaker 4: like everybody else, you know, I talk to people every
Speaker 4: day all day, and so I talk to people. You know,
Speaker 4: we have a bunch of meetings one after the other. End.
Speaker 4: I have teams coming to me with either regularly scheduled
Speaker 4: meetings or meetings that have been requested to discuss a
Speaker 4: certain topic. And you know, we just go through is
Speaker 4: there a whiteboard? Well right now in the age of Zoom,
Speaker 4: I guess still. You know, we have a globally distributed
Speaker 4: team and so a lot of our people are not
Speaker 4: in the same office, and so we use virtual whiteboards
Speaker 4: like everybody else. But I would say, you know, one
Speaker 4: of the things that I tried to do while joining Golma,
Speaker 4: which was part of sort of the cultural agen that
Speaker 4: was emphasizing the importance of narratives and written world versus
Speaker 4: you know, PowerPoint and talking. Okay, so, which is kind
Speaker 4: of what I learned that Amazon over the years. Okay,
Speaker 4: all right, w I was a REDWS and one of
Speaker 4: the things you learned there as soon as you join Amazon,
Speaker 4: in any part of Amazon, like the first few meetings
Speaker 4: are kind of shocking because nobody talks. Everybody starts reading.
Speaker 4: You start reading for like sometimes thirty minutes or forty
Speaker 4: five minutes, and if you're the author of the document,
Speaker 4: you're just sitting there basically, and you just try to
Speaker 4: look at people's faces and understand what they think about
Speaker 4: your document. And sometimes, you know, if you're with Jeff
Speaker 4: Bezos or others, you know, at that time it can
Speaker 4: be pretty pretty terrifying. And so this kind of shift
Speaker 4: from a culture of people talk, people comment on a PowerPoint,
Speaker 4: and the discussion sometimes get you know, driven by who
Speaker 4: has the stronger personality versus, you know, who has the
Speaker 4: greatest ideas. One of the things that I try to
Speaker 4: change is that a lot of the meetings that we
Speaker 4: do today actually start the same way by reading a document.
Speaker 4: So I now read a lot of documents like I
Speaker 4: used to in Amazon. You know, I would say maybe
Speaker 4: thirty forty percent of the meeting are starting that way,
Speaker 4: and I think people love it because it breaks the
Speaker 4: barrier of language for someone like me, that English is
Speaker 4: obviously not my first language, breaks the Sometimes some of
Speaker 4: the people are more shy than others, et cetera. So
Speaker 4: people see that as a mechanism for inclusion. So back
Speaker 4: to your question, let's say thirty forty percent of my
Speaker 4: meetings actually now start by us reading a document together
Speaker 4: and then commenting on that and making decisions.
Speaker 3: Can I just say, Tracy, I've always thought more meetings
Speaker 3: you should start with just reading. Because you go to
Speaker 3: you hear like a quarterly call or a FED event,
Speaker 3: and someone just reads out of prepared text. It's like,
Speaker 3: just let everyone read it and just jump straight into like,
Speaker 3: let everyone do the reading first.
Speaker 2: You don't need someone.
Speaker 3: Standing up there talking about what's on a written piece
Speaker 3: of paper somewhere.
Speaker 1: Anyway, I agree that we could reduce the time of meetings. Yes, okay,
Speaker 1: So speaking of meetings and the decision making process, then
Speaker 1: talk to us about how Goldman Sachs decided to approach
Speaker 1: generative AI. What was the decision making process? Like there
Speaker 1: the development process, and you know, we'll get to what
Speaker 1: you're developing, but like, how did you initially approach it?
Speaker 4: So I think our initial approach was really to realize
Speaker 4: that there were so many more things that we didn't
Speaker 4: know compared to the things that we knew, because it's
Speaker 4: a really new thing, and even for companies like us
Speaker 4: that have been working on machine learning and traditionally I
Speaker 4: for literally decades, this felt like a very different thing.
Speaker 1: What sort of timeframe are we talking about? Like, was
Speaker 1: there a sort of like big realization that this is
Speaker 1: something that we need to focus on.
Speaker 4: Yes, because I was lucky enough that I got into
Speaker 4: the very very early version of GPT, even before it
Speaker 4: was called chat GIBT. So the very first version was
Speaker 4: essentially completing a sentence. It wasn't even allowing you to
Speaker 4: do interactive chat. You would just paste a text and
Speaker 4: that will just complete that text. And so I started
Speaker 4: to do that with a bunch of stuff, and then
Speaker 4: I was saying that the quality which this will continue
Speaker 4: was pretty much indistinguishable with the part that you actually
Speaker 4: put in that. And so we started to obviously talk
Speaker 4: between ourselves but also among other people in the industry,
Speaker 4: and we all realized very soon that this would be
Speaker 4: something very different, but be also something that could have
Speaker 4: a pretty profound impact in what we do. Because at
Speaker 4: the end of the day, we are a purely digital business.
Speaker 4: We don't bend metal, we don't you know, like use
Speaker 4: high temperatures. We don't really have physics. So it's all
Speaker 4: about how we service our clients. It's all about how
Speaker 4: smart we are. It's all about how we can process
Speaker 4: incredible amount of information. It's all about, you know, how
Speaker 4: we analyze data in a very sometimes opinionated way. We
Speaker 4: form our own views on the market, we form our
Speaker 4: views of investments, et cetera. And so given that this
Speaker 4: AI showed very early sign of being able to synthesize
Speaker 4: and summarize very complex set of information but also identify patterns,
Speaker 4: we thought that could be something that we definitely need
Speaker 4: to pay attention to. So given that, one of the
Speaker 4: things that we decided to do very early on was
Speaker 4: to put a structure and I can say that more
Speaker 4: about that, put a structure around this so that we
Speaker 4: could experiment but in a sort of safe and controlled way.
Speaker 1: Right, So you decided to develop your own Goldman Sachs
Speaker 1: AI model versus you know, use a chat, GPT or
Speaker 1: clod or getting something off the show.
Speaker 4: Actually, initially we kind of thought about that, but then
Speaker 4: very quickly. We decided that our time was spent much
Speaker 4: better with using existing models, which by the way, we're
Speaker 4: iterating really really quickly, but then put them in a
Speaker 4: condition so that they would be safe to use and
Speaker 4: also they would actually give us the most reliable information,
Speaker 4: because taken as they are, you can't just drop a
Speaker 4: model in an environment like Goldman and then, like you know,
Speaker 4: to your earlier point of a one percent in accuracy,
Speaker 4: zero point one percent in accuracy completely an acceptable class.
Speaker 4: There are a lot of potential issues related to you know,
Speaker 4: what data has it been used to train? And you know,
Speaker 4: there is a lot of uncertainty with regards to you know,
Speaker 4: like what are the boundaries between what you can safely
Speaker 4: use and what you can And so what we decided
Speaker 4: to do was instead to build a platform around the model.
Speaker 4: So think of that almost as if you had a
Speaker 4: nuclear reactor. You know that now you have invented fission
Speaker 4: or fusion, and there is a lot of power that
Speaker 4: can be generated from that, but then you need to
Speaker 4: contain it and direct it in a certain way. And
Speaker 4: so we build this GSAI platform, which essentially takes a
Speaker 4: variety of models that we select, puts them in the
Speaker 4: condition of being completely segregated and completely secluded and completely
Speaker 4: safe from an information a security standpoint. Abstract some of
Speaker 4: the ways to use the model, so that our developers
Speaker 4: can use the models interchangeably, and then creates a set
Speaker 4: of standardized way, for example, improve the accuracy using retrieval,
Speaker 4: a granted generation, access external or internal data sources, applying
Speaker 4: entitlement so that someone is on the private side, you know,
Speaker 4: I've got to see different information that someone is on
Speaker 4: the public side. And then on top of that, build
Speaker 4: a developer environment so that people will very easily be
Speaker 4: able to embed that AI in their own applications. So
Speaker 4: imagine this, we got a great engine and we decided
Speaker 4: to build a great car around that.
Speaker 2: What are you putting in the model?
Speaker 3: Because I have to imagine at a bank like Goldman,
Speaker 3: you know, you have a lot of data, but you
Speaker 3: must have just an extraordinary amount of unstructured data. There's
Speaker 3: conversations that bankers have with clients. There's other sort of meeting,
Speaker 3: the meetings you have, and there's words that are said
Speaker 3: during that meeting that could be synthesized in some way.
Speaker 3: In these early iterations, you know, I upload a conference
Speaker 3: called transcript and I ask a question, what do you
Speaker 3: upload it? What is the unstructured data that you have
Speaker 3: or the questions or these yeah, what are you what
Speaker 3: are you putting into it from your reams of knowledge
Speaker 3: that you must have internally.
Speaker 4: So one of the first things that we did was
Speaker 4: use the platform and the models to extract information from
Speaker 4: publicly available documents. That's kind of the safest way public
Speaker 4: filing all the case or the queues and you know,
Speaker 4: and obviously earnings, and put our bankers in a condition
Speaker 4: to be able to ask very very sophisticated multi dimensional
Speaker 4: questions around what was reported, cross refit with previous reports,
Speaker 4: cross refit with any announcement, any earnings, called transcripts, all
Speaker 4: things that are out there but just are difficult to
Speaker 4: bring together. And so that as a involved into a
Speaker 4: tool that physically we use and we're rolling it out
Speaker 4: right now as an assistant to our bankers so that
Speaker 4: they can you know, service their client or answer client
Speaker 4: questions or even their wrong questions. In a time there
Speaker 4: is a fraction of what you used to take even
Speaker 4: generate documents that then can be you know, shared the
Speaker 4: clients and so on and so forth. And obviously we
Speaker 4: always have as a rule, like when you drive a
Speaker 4: car that has some autonomous capability, that you always keep
Speaker 4: the hands on the wheel. Our rule is that there
Speaker 4: always needs to be a human in the loop. Okay,
Speaker 4: And so the way that works is actually interesting because
Speaker 4: we found out that you can't just shove something into
Speaker 4: a model and then pretend that the model is going
Speaker 4: to give you the answer right away. Why well, because models,
Speaker 4: by themselves, you know, they essentially apply a stochastic or
Speaker 4: a statistical way to understand what is the next world
Speaker 4: that they need to say. So, no matter how good
Speaker 4: is the material that you put in, there's always going
Speaker 4: to be some level of variability. There is almost like
Speaker 4: the intersection between the documents that you insert and what
Speaker 4: is I call it like the shadow of all the
Speaker 4: knowledge of all the things that the model has seen before.
Speaker 4: And so we really perfected this. You know, there are
Speaker 4: two techniques that are widely used to improve the accuracy
Speaker 4: of the answers. One is working on the way those
Speaker 4: models represent knowledge, which is called embeddings technically, and the
Speaker 4: concept of embeddings by the way, everybody talks about embeddings,
Speaker 4: but then for very few people actually it took me
Speaker 4: a while to understand that well. And embedding is simply
Speaker 4: a way for the model to parameterize and create a
Speaker 4: description of what they're seeing. So if I see a phone,
Speaker 4: for example, in front of me, the embeddings of a
Speaker 4: phone could be it's a piece of electronic Yes, one,
Speaker 4: it's definitely a piece of electronics. It's edible. Zero. You
Speaker 4: can't really eat it, you know, And then you have
Speaker 4: all these parameters. Is almost like twenty questions. I give
Speaker 4: you all these questions and then you finally understand that
Speaker 4: it's a phone, and that's what the embeddings is almost
Speaker 4: like the twenty questions of the reality instead of twenty
Speaker 4: is like twenty twenty thousands. And then you have DRAG,
Speaker 4: which is the retrieval augmented generation, which is actually interesting
Speaker 4: because you tell the model that instead of using its
Speaker 4: on internal knowledge in order to give you an answer,
Speaker 4: which sometimes, as I said, is like a representation of reality,
Speaker 4: but it's often not accurate, you point them to the
Speaker 4: right sections of the document that actually is more likely
Speaker 4: to answer your question. Okay, and that's the key. It
Speaker 4: needs to point to the right sections and then you
Speaker 4: get the citations back. So that took a lot of effort.
Speaker 4: But we're using that in many many cases because then
Speaker 4: we expanded the use case from purely like banker assistant
Speaker 4: in a way to more like okay, document management. You know,
Speaker 4: we process millions of documents. Think of that credit confirmation
Speaker 4: implements confirmation. Every document has a task called entity strauction.
Speaker 4: So you need to extract stuff from the document and
Speaker 4: then digitize it and then model it in a certain way.
Speaker 4: And so the use of general TVii there does a
Speaker 4: great job at extracting information. And this is an interesting
Speaker 4: concept because you don't have to actually tell a fixed pattern.
Speaker 4: You can just say, give a lot of examples, and
Speaker 4: then the AI will figure out from that pattern. One
Speaker 4: of my favorite example is the following. Let's say that
Speaker 4: my phone number is five five three two one three
Speaker 4: h five oh, and someone writes in the document instead
Speaker 4: of with zero rights an oh. Okay. You can test
Speaker 4: yourself even with GPT, if you give a number with
Speaker 4: an O instead of zero, and you ask GPT, what's
Speaker 4: likely wrong with this entity? GPT is gonna tell you, well,
Speaker 4: it looks like a phone number that is an all,
Speaker 4: which general is not in phone numbers. Most likely this
Speaker 4: is the correct phone number. Now, nobody has written software
Speaker 4: to do a pattern match in there. And imagine if
Speaker 4: in the tradition, in traditional way of doing antity instruction,
Speaker 4: there were developers that were writing rules. They were saying, okay, numbers,
Speaker 4: it needs to be ten digits and blah blah blah.
Speaker 4: The AI figures.
Speaker 2: Out their own rules.
Speaker 4: That are the most likely. So this is the key thing.
Speaker 4: It has common sense. And that common sense when you're
Speaker 4: dealing with millions of documents that contain all bunch of
Speaker 4: ways that you must might have written those things, and
Speaker 4: imagine the complexity of all the rules that you need
Speaker 4: to write. And every bank has the same problem. This
Speaker 4: simplifies things tremendously because it's able to figure out what's
Speaker 4: most likely by itself. And so that thing evolved into
Speaker 4: a tremendous time saving for everybody in the bank that
Speaker 4: has to do with the workflow documents. And so that
Speaker 4: was a very interesting finding that we did early on.
Speaker 4: And so again to summarize more, those are raw material
Speaker 4: of intelligence. You know you need to somehow direct them,
Speaker 4: you need to guide them, you need to instruct them,
Speaker 4: you need to put them in an environment that actually
Speaker 4: gets the most out of that, and that's what we've
Speaker 4: been focusing on.
Speaker 1: So going back to the analogy that you used previously,
Speaker 1: this idea of a nuclear reactor and sort of building
Speaker 1: the containment casing or the protective casing around it. I
Speaker 1: imagine one of the complications of being Goldman Sachs and
Speaker 1: working with AI is that you're a regulated financial entity.
Speaker 1: How does that added complexity affect your use of AI.
Speaker 1: Are there additional data considerations or additional infosec considerations.
Speaker 4: I think that's a great question, because obviously we live
Speaker 4: in a regulated world, and in fact, I have to
Speaker 4: tell you that in this case, regulation actually helps us
Speaker 4: think through all the possible unknown now, something that, as
Speaker 4: I said, is something that is still largely something that
Speaker 4: nobody really completely understands. And so what we did was
Speaker 4: to put but governance around the usage of the models
Speaker 4: and also governance with regards to the use cases that
Speaker 4: we can implement on the models. Every bank has a
Speaker 4: function called model risk, which, in the traditional sense, a
Speaker 4: model is any decision or any algorithm that is running
Speaker 4: automatically to do for example, pricing or you know, there
Speaker 4: is a lot of that tradition in every bank risk calculation, etc.
Speaker 4: So that's the traditional model risk. We use that very
Speaker 4: well established pattern. That is also you know, that has
Speaker 4: its own second and third line like controls and supervision
Speaker 4: also to validate what we do on the AI side.
Speaker 4: So there is a governance part which we really set
Speaker 4: up very early on. We have an AI committee that
Speaker 4: looks at the business case should we do this? And
Speaker 4: then we have an AI control and risk committee that
Speaker 4: looks at, okay, how are we going to do that?
Speaker 4: And then the two of them need to actually come
Speaker 4: together before we can release a use case. And then
Speaker 4: of course we did a lot of work with regards
Speaker 4: to the let's say accuracy lineage and in a way,
Speaker 4: the way you connect the output to where does the
Speaker 4: data come from and who can actually see that what
Speaker 4: we call entitlements, and we did that in lockstep with
Speaker 4: the regulators, so that I think, you know, you know,
Speaker 4: in a world, I think we put a sort of
Speaker 4: what we like to call responsible AI first since the
Speaker 4: very beginning, and it really helped us. The fact that
Speaker 4: you know, we embedded all those controls into a single platform.
Speaker 4: This is how our people use AI inside the inside goal.
Speaker 1: This is something I'm really interested in just from a
Speaker 1: technical perspective, But can you talk a little bit more
Speaker 1: about that interoperability aspect. So you have a pool of
Speaker 1: data that is gold pins that you presumably don't really
Speaker 1: want to share with outside entities, So how do you
Speaker 1: plug that into an AI model if you're working with
Speaker 1: you know, Chat, GPT or clod or something like that.
Speaker 4: So there are two ways that we do that. We
Speaker 4: use the sort of a large proprietary models in a
Speaker 4: way that we worked with Microsoft, we work on Google.
Speaker 4: We have very strong partnerships, so that essentially there are
Speaker 4: controls that guarantee that nobody has access to the data
Speaker 4: that we put into the model, that the data leaves
Speaker 4: no side effects, so it's not saved anywhere, it's the
Speaker 4: only stays in memory. The model is completely stateless, meaning
Speaker 4: that the state of the model doesn't change after the
Speaker 4: data comes through, so there is no training, there is
Speaker 4: nothing down on that data. And also that operator access
Speaker 4: meaning who can actually access the memory or those machines
Speaker 4: is restricted and controlled and needs to be agreed with us.
Speaker 4: So imagine secure in putting a vault around those models.
Speaker 4: But even then, what's really really sort of secret, source, proprietory, etc.
Speaker 4: We like to use also different approach to use open
Speaker 4: source models that we can run on our own environment. Okay,
Speaker 4: and we like a lot of open source models. I
Speaker 4: have to say that. One we particularly like Islama and
Speaker 4: actually Lama tree and Lama tree point one especially as.
Speaker 2: No one developed by Facebook.
Speaker 4: Oh yeah, so they recently announced Lama three point one,
Speaker 4: which has a version that is four hundred and five
Speaker 4: million billion parameters. So it's pretty large and it seems
Speaker 4: to be performing. You know, the gap with those big
Speaker 4: fundational models is now very very narrow. So for that,
Speaker 4: we run it in our own sort of a private cloud,
Speaker 4: call it that way, with GPUs that we own, and
Speaker 4: that we train it with data that stays in that environment.
Speaker 4: So imagine that. You know, our approach is okay, there
Speaker 4: is a sort of arrating of sensitivity of this data.
Speaker 4: Every data needs to be protected. Therefore we use those
Speaker 4: safeties all throughout regardless. But then for the super super
Speaker 4: super secret stuff, you know, we like to do it
Speaker 4: in our own embod.
Speaker 3: Since you're talking about building your own environment, and this
Speaker 3: is something we've talked a lot about on the podcast.
Speaker 3: Hardware constraints, energy constraints, things like that, how does that
Speaker 3: manifest in your world some of these physical, real world
Speaker 3: constraints to building out the compute platform at Goldman sax Well.
Speaker 4: Initially we thought maybe we can host those GPUs in
Speaker 4: our own data centers, and then immediately you run into
Speaker 4: considerations such as a first of all, they develop a
Speaker 4: lot of heat. Secondly, they consume a lot of power.
Speaker 4: Tree there is a decent chance that they might fail
Speaker 4: because you know, of all those considerations if you're not
Speaker 4: properly addressed. And then d they need very special for example,
Speaker 4: interconnect and high speed bandwidth between them. And so the
Speaker 4: decision what we ended up doing is actually to have
Speaker 4: them hosted into some of the hyperscalers that we use,
Speaker 4: but use them in their own virtual private clouds. So
Speaker 4: those racks are basically only ours. And if you're asking
Speaker 4: me the more general question, which is, hey, where is
Speaker 4: the world going with regards of that? Okay, so right
Speaker 4: now there are two really rapidly competing forces. One is
Speaker 4: pushing towards more and more consumption and one is pushing
Speaker 4: for more and more optimization. Okay, and I can talk
Speaker 4: about that for a couple of minutes. For the more consumption,
Speaker 4: I mean, really the two dimensions for scaling a model
Speaker 4: is one of the most important. One is obviously the
Speaker 4: size of the prompt or the context. Okay, and there
Speaker 4: is pretty good evidence that the larger the context, which
Speaker 4: is really like the memory of those models, and the
Speaker 4: more you can get out in terms of the ability
Speaker 4: to reason on your data. That has already gone up
Speaker 4: from thousands to tens of thousands to now millions. And
Speaker 4: there is a prediction, you know, you heard some very
Speaker 4: prominent people saying that there could be the trillion prompt
Speaker 4: and the power scales quadratically with the prompt, so that
Speaker 4: points to a consumption of energy and GPU power which
Speaker 4: is going to continue to raise exponentially. At the same time,
Speaker 4: we've seen great results with optimization techniques such as quantitization,
Speaker 4: reducing from sixteen bits to eight bit to four bit precision,
Speaker 4: having even smaller models using what's called window that which
Speaker 4: means that you know that you can only pay more
Speaker 4: attention to some of the parts of the context intell
Speaker 4: of all of it, and so you need a smaller one.
Speaker 4: And so I'm seeing those two kind of going into
Speaker 4: two opposite directions. It's going to be very interesting to
Speaker 4: see how that evolves. I would say for the short term.
Speaker 4: I see that definitely that trend is going to continue
Speaker 4: to go up. And one of the things that fascinates
Speaker 4: me the most is that from one version to another,
Speaker 4: the most striking difference is the ability to reason and
Speaker 4: the ability to actually come up with logical step by
Speaker 4: step instructions or step by step chains of thought of
Speaker 4: what the output is going to be. So we decided, okay,
Speaker 4: first of all, we need to get access to the
Speaker 4: most powerful GPUs, secondary we need to host them into
Speaker 4: an environment that actually allows for the most optimal functioning
Speaker 4: in terms of bandit, in terms of power consumption, etc.
Speaker 4: And then at the same time, we've been focusing a
Speaker 4: lot on optimizing the algorithm so that you know, we
Speaker 4: can really got we could really get the most out
Speaker 4: of that.
Speaker 1: Just to press you on this point, what are the
Speaker 1: conversations actually like with cloud providers at the moment when
Speaker 1: you're trying to get more compute or more space, more racks, whatever.
Speaker 1: Is it maybe different for you because you were at AWS.
Speaker 1: Maybe you can just call someone up there and be like,
Speaker 1: we would like some more servers, or have you found
Speaker 1: yourselves at times maybe limited in what you can do
Speaker 1: by the amount of power available to you.
Speaker 4: Well, I wish that would be the case, but I
Speaker 4: cannot just pick up the phone and get whatever I want.
Speaker 4: But I think so far. I mean obviously because we
Speaker 4: are a really good client of those companies in general,
Speaker 4: but also because we've been very selective in the use
Speaker 4: cases that we put in production. I have to say,
Speaker 4: like I said before, think about that, if you look
Speaker 4: at the consumption of resources today, those who consume more
Speaker 4: resources are people that actually do the training of their
Speaker 4: own models. Okay, and it initially everybody was trying to
Speaker 4: do full training from scratch, which was taken like the
Speaker 4: absolutely if that's one hundred, we do fine tuning, which
Speaker 4: is adaptation of existing models that could be one to
Speaker 4: one hundred or less in terms of consumption or resources.
Speaker 4: So because of the techniques they were using, and because
Speaker 4: of the fact that we decided to really focus on
Speaker 4: fine tuning or RAG versus full training, we haven't really
Speaker 4: hit any caps. And also have to be honest, you know,
Speaker 4: we bought our GPUs pretty well early, so probably there
Speaker 4: wasn't as much craziness as there is today, and so
Speaker 4: that's turned out probably to be a good idea.
Speaker 2: You know, in videos huge.
Speaker 3: Everyone would like to have some of in Video's market
Speaker 3: cap be their market cap. I have offering some cheaper product.
Speaker 3: We interviewed some guys who have a semiconductor started that's
Speaker 3: just going to be LLLM focused startups. We know that Google,
Speaker 3: for example, has TPUs their own chips. Can you envision
Speaker 3: as a roadmap some alternative where GPUs are not the
Speaker 3: dominant hardware for AI?
Speaker 4: Well, that's literally like you know the trillion dollar question.
Speaker 2: Yeah, well that's I'm asking you.
Speaker 4: Yeah, but I'm not an analyst and I'm just a technogy.
Speaker 4: Remember I'm the guy that makes sure that I.
Speaker 3: Would say, you're probably a better person to ask than
Speaker 3: an analyst because you're actually the one who's going to
Speaker 3: be making So I'm.
Speaker 4: Okay, so they're going to ask it to you. So
Speaker 4: you have to distinguish between There are actually two dimensions
Speaker 4: that we need to consider. One is training and the
Speaker 4: other one is inferenced. Okay, that's the first dichotomy. For training.
Speaker 4: At the moment, there's most likely nothing better than GPU's okay,
Speaker 4: because when you train a model, the software or Pythons
Speaker 4: or whatever framework needs to see all your GPUs as one.
Speaker 4: As a cluster, and it's not just the GPU itself,
Speaker 4: but it's the what Nvidia has been doing a great
Speaker 4: job at is actually to make them work in unison
Speaker 4: with the virtualization software called Kuda, which runs on and
Speaker 4: video GPUs, which is a extraordinary piece of software and
Speaker 4: it became the standard for that. And also because you know,
Speaker 4: the performance premium that you have on those GPUs when
Speaker 4: you're trying to train those incredibly large models is something
Speaker 4: that you really really want. And so the training part,
Speaker 4: I'm pretty sure that it's going to be dominated by
Speaker 4: GPUs for a while. But then you know, as those
Speaker 4: models get used, obviously the pendulum swings towards inference, which
Speaker 4: is the actual Now you have a model which is
Speaker 4: a bunch of weights and you just need to calculate
Speaker 4: a bunch of matrix multiplications on that. I think accelerators
Speaker 4: and specialized chips are actually going to have a really
Speaker 4: big role to play. So you may imagine that you
Speaker 4: go from a world where everybody builds the cars and
Speaker 4: not too many people drive the cars to a world
Speaker 4: where most people are going to drive cars. And then
Speaker 4: there is another two dimensions, which is models that are
Speaker 4: hosted by the client and models that are hosted by
Speaker 4: a hyperscale. So, as you know today, I can take
Speaker 4: a model like Lamma, I can put it in my
Speaker 4: own environ, I can run it on a MacBook, or
Speaker 4: I can run it in my own data center and
Speaker 4: with my own GPUs. And given that I'm used to GPUs,
Speaker 4: given that those are the ones that we can buy,
Speaker 4: given that Kuda is what developers know, etc. I'm most
Speaker 4: likely going to use that. That's a good part for
Speaker 4: Nvidia for that. But then there is another way to
Speaker 4: use those models, which is to have someone host them
Speaker 4: for me and I just access them to an API.
Speaker 4: That's what services like Amazon Bedrock does. You basically choose
Speaker 4: your own model and then you serve it through them.
Speaker 4: When you do that, you don't really know what's underneath.
Speaker 4: You don't know if it's a VP, or if it
Speaker 4: is an accelerator, if it is Amazon's own chips or
Speaker 4: Google's own chips, etc. So now the real question, that's
Speaker 4: why the trillion dollar question is are most people going
Speaker 4: to use those models through hosted environments where the hyperscaler
Speaker 4: will have a lot of freedom with regards to what
Speaker 4: they use underneath, and most likely they will vertically integrate
Speaker 4: or are they going to use them you know, themselves
Speaker 4: in a more more like you know, in a self
Speaker 4: service way, And in that case it's less likely that
Speaker 4: those accelerators are going to dominate. We currently are in
Speaker 4: a sort of a you know, balanced way because we
Speaker 4: have our own that we use like I described, and
Speaker 4: also we use you know, the hosted models. And so
Speaker 4: where is this going to go? It's hard to say,
Speaker 4: because I think it depends on the evolution of the models,
Speaker 4: and it depends which models are going to be made
Speaker 4: available as an open source that you can actually host yourself.
Speaker 4: And I think right now one of the greatest questions
Speaker 4: is are the open source models are going to be
Speaker 4: in absolutely on parer alternative to the to the hosted model,
Speaker 4: to the to the foundational proprietary models, and that given
Speaker 4: Glama three point one, that answer seems to be more likely.
Speaker 1: Yes, I had a question about this actually, which is
Speaker 1: do you think Wall Street's attitudes towards open source have
Speaker 1: changed over time? And the reason I ask is because
Speaker 1: nowadays it seems like a fact of life. Everyone uses
Speaker 1: open source, whether you're a Goldman or somewhere else. But
Speaker 1: I remember, you know, like back in as recently as
Speaker 1: like twenty twelve. I remember Deutsche Bank had like this
Speaker 1: open source project called the Loadstone Foundation, where they were like, oh,
Speaker 1: we should all stop wasting our own resources developing our
Speaker 1: own code and our own software. We should all pool
Speaker 1: our resources together and do open source. And they had
Speaker 1: to actually lobby. It was unsuccessful ultimately, but they were
Speaker 1: trying to get all the banks on Wall Street to
Speaker 1: work together for open source. Nowadays, it seems like there's
Speaker 1: been this significant cultural shift, it's not even a question.
Speaker 4: So in general, my direction, my guide as to you know,
Speaker 4: my team is a don't build anything unless you have to.
Speaker 4: Don't think that just because you're a smart person you
Speaker 4: can build software better than anybody else. Maybe you can,
Speaker 4: but it's a good thing that we focus on building
Speaker 4: things that are actually differentiating for us. And then I
Speaker 4: think the use of open source software, which we very
Speaker 4: much endorse, is also really good hedge with regards to
Speaker 4: you know, which vendors to use, because it really heavily
Speaker 4: reduces the vendor lock in. Of course, open source software,
Speaker 4: as you know, is a tremendous long tail. There's millions
Speaker 4: of that, and so I think there are best practices
Speaker 4: around the use of open source, and those best practices are,
Speaker 4: you know, like you know you need to run reviews
Speaker 4: on open source tech or tech risk reviews or security
Speaker 4: reviews or anything as I've almost built it yourself. And
Speaker 4: then secondly tending to concentrate on the larger, very well
Speaker 4: supported by the community type of open source. And so
Speaker 4: my philosophy is yes to open source, but then you
Speaker 4: need to own it in in truest way because you
Speaker 4: are actually going to be generally the one that actually
Speaker 4: needs to support that as or really building knowledge around that.
Speaker 1: And now you can ask AI to run the code
Speaker 1: for you and check it.
Speaker 4: For yeah, okay. That of course leads to probably what
Speaker 4: if you ask everybody where did you get so far?
Speaker 4: The biggest bank for the back for AI? Most CIOs
Speaker 4: are going to tell you on developer productivity. And I
Speaker 4: think it's something that for us was the first project
Speaker 4: that we actually expanded at scale. I have to say
Speaker 4: that today virtually every developer in Goma SACS is equipped
Speaker 4: to with generative coding tools, and you know we have
Speaker 4: twelve thousand of that. So we didn't enable yet the
Speaker 4: ones that are using our own proprietary language called slang,
Speaker 4: but everybody else has an AI tool and the resulso
Speaker 4: be pretty extraordinary.
Speaker 2: How do you measure that? What are what are some numbers?
Speaker 2: Or how would you describe the right?
Speaker 4: So we measure it according to a number of metrics,
Speaker 4: such as the time that it takes from let's say
Speaker 4: when you start the sprint, when you actually commit the code,
Speaker 4: or when you complete your task. We measure it by
Speaker 4: number of commits, meaning how many times you actually put
Speaker 4: code into production. We measure it by a number of defects,
Speaker 4: which in this case is like, for example, deployment related errors.
Speaker 4: So there are more like velocity and quality metrics. At
Speaker 4: the same time, we have seen a wide range ranging
Speaker 4: from ten to forty percent productivity increase. I would say
Speaker 4: that today we are probably on average seeing twenty percent. Now,
Speaker 4: developers don't spend one hundred percent of their time coding.
Speaker 4: They maybe spend fifty percent of their time coding. So
Speaker 4: your question is what are they doing with half of
Speaker 4: their times where there is a lot of other activities
Speaker 4: such as documenting code, such as doing deployment, doing deployment scripts,
Speaker 4: doing you know, buntio tests, et cetera, et cetera. So
Speaker 4: what's called generally the software development life cycle. Okay, and
Speaker 4: so we see net of ten percent. But then the
Speaker 4: cool thing is that those AIS and the things that
Speaker 4: we're building around that are starting to go beyond coding.
Speaker 4: They're starting to help you write the right tests, write
Speaker 4: the right documentation. They are even figure out algorithms or
Speaker 4: even for example, reducing or minimizing the likelihood of deployment
Speaker 4: issues writing deployment scripts for you. So as that expands,
Speaker 4: we're going to be closer to one hundred percent, and
Speaker 4: therefore we're going to be closer probably to twenty percent,
Speaker 4: which you know, for an organization of our side, is
Speaker 4: a pretty massive efficiency play.
Speaker 3: Can I ask a question about hiring developers? So I've
Speaker 3: probably read one hundred articles over the years about Wall
Speaker 3: Street competing with tech companies to hire developers, like, oh,
Speaker 3: they got a ping pong Lloyd Blank.
Speaker 1: Fine used to say, they are a technology company.
Speaker 3: Yeah, you gotta have your ping pong tables and your
Speaker 3: free lunches and let people are sneakers and I have
Speaker 3: all that stuff. But now it seems with AI, there's
Speaker 3: a number of people interested in who are truly believing
Speaker 3: that within a few years they might build the digital
Speaker 3: god that's ten thousand times smarter than any human, and
Speaker 3: that they approach the task with messianic fervor. And I
Speaker 3: imagine it, right if you're at Goldman and you're trying
Speaker 3: to help a banker answer a question to a client
Speaker 3: about something in the chemical industry, like maybe that's not
Speaker 3: like the thing that gets you out of bed the way,
Speaker 3: sort of like metaphysical realms about what is the nature
Speaker 3: of consciousness and things like that that people talk. Does
Speaker 3: that present any challenges or anything when trying to hire
Speaker 3: talented a developers.
Speaker 4: I think developers love to solve real problems. And one
Speaker 4: of the things also that attracted me in the first place,
Speaker 4: Not that it matters, but I'm saying, you know, I
Speaker 4: tell you my own personal experience is that working in
Speaker 4: a technology company is absolutely fantastic, but you're always like
Speaker 4: one step removed from the business or from the application.
Speaker 4: So I have to you know, let's say you are
Speaker 4: the bank and I'm the technology company. I need to
Speaker 4: sell you a tool that then you're going to use
Speaker 4: to run your business or improve your business. We are
Speaker 4: kind of one degree of separation. Less I were right
Speaker 4: there in a digital business there is fast, huge amounts
Speaker 4: of data, huge amounts of flaws, immediate results, and that's
Speaker 4: kind of addictive. And so developers, especially when a AIS
Speaker 4: are starting to do all those magical things that we're
Speaker 4: talking about, you know, they can see the impact on
Speaker 4: the business right away, and then I think is kind
Speaker 4: of attracting a lot of people. In fact, that there
Speaker 4: is more and more people that are moving into the
Speaker 4: industries oil and gas, transportation, chemical, medical, finance because you know,
Speaker 4: this is new and there's nothing more exciting than seeing
Speaker 4: it in action. And so there is so much action
Speaker 4: going on that I think is actually really really interesting.
Speaker 4: I think another question that maybe you haven't asked me,
Speaker 4: but it's kind of part of this question, is what
Speaker 4: kind of developers? How is the profession of being a
Speaker 4: developers actually changed?
Speaker 1: Oh wait, I had a related question. It's not quite
Speaker 1: that question, but you can certainly answer that too. But Okay,
Speaker 1: to my knowledge, Goldman Sachs doesn't have a job title
Speaker 1: specifically with the words prompt engineer in it. So, looking
Speaker 1: at the impact of AI on your business overall, is
Speaker 1: AI a net hiring positive or a net hiring negative
Speaker 1: for gold Men's employees overall?
Speaker 4: Well, meaning, are we going to hire more or less development?
Speaker 1: Yeah, it doesn't lead to more jobs because you're doing
Speaker 1: more things and productivity increases. Or does it lead to
Speaker 1: fewer jobs because now you can automate a bunch of stuffs.
Speaker 4: Well, listen, there is so many things that we would
Speaker 4: like to do if we had more resources that I
Speaker 4: think this is going to be leading to more things
Speaker 4: that we can do. You know, some people tell me sometimes,
Speaker 4: so you're gonna maybe hire less or have less developers.
Speaker 4: I don't know. I've been in it quote and quote
Speaker 4: for like literally almost forty years, and I've never ever
Speaker 4: seen that go down. But I've seen inflection points where
Speaker 4: you can actually get developers to do way more and
Speaker 4: worry about way less. There is not related to a
Speaker 4: business outcome, and so I think it's more like how
Speaker 4: the profession is going to change. In my opinion, we're
Speaker 4: going to be less low level and more Hey, I
Speaker 4: need to really understand the business problem. Hey, I really
Speaker 4: need to think outcome driven. Ay, I need to have
Speaker 4: a crisp mental model and I need to be able
Speaker 4: to describe it in words. So the profession is going
Speaker 4: to change, and there are tasks that I think are
Speaker 4: so repetitive that the automation of those is actually going
Speaker 4: to help developers, you know, really kind of feeling really
Speaker 4: really connected with the business and with the strategy, and
Speaker 4: that will attract people that are generally curious, that are
Speaker 4: generally interested in understanding what we actually do. So the
Speaker 4: focus kind of shifts from the how to do what
Speaker 4: and to the why, which is really kind of the heart.
Speaker 4: Or think of this evolution of technology over the years
Speaker 4: from the back office of it, which doesn't even know
Speaker 4: what you're doing, but as long as your monitor is
Speaker 4: actually working to hey, I'm actually able to take a
Speaker 4: business problem and break it down into pieces that then
Speaker 4: even an AI can write code for. So to your
Speaker 4: specific question, I think this might maybe potentially for some
Speaker 4: companies are going to try to realize some of those
Speaker 4: efficiencies by curbing the growth or even sometimes reducing it.
Speaker 4: For companies like us that are extremely competitive, for companies
Speaker 4: that have lots of ambition, this race at the end
Speaker 4: of the day, and I think we're going to go
Speaker 4: for you know, trying to get even more out of
Speaker 4: our developers and actually like you know, trying to turn
Speaker 4: them more into something that makes them feel super super connected.
Speaker 4: To the business.
Speaker 3: What about non developer roles, non tech roles, And you know, again,
Speaker 3: I guess a company like Goldman doesn't have you know,
Speaker 3: probably a lot of like low level customers support things
Speaker 3: for in a window is like oh, I need to
Speaker 3: change my plane ticket, et cetera. But you know, a
Speaker 3: lot of modern work is essentially just answering somebody's basic question.
Speaker 3: Are the roles within a bank that are going to
Speaker 3: either fundamentally change or go away due to sort of
Speaker 3: agentic or generative AI.
Speaker 4: I think a lot of the work there is about
Speaker 4: content production or content summarization will actually be streamlined quite
Speaker 4: a bit, like, for example, taking an earnings report, making
Speaker 4: it into ten different sauces in order to wear for
Speaker 4: different channels of distribution. Here's the one for internal people,
Speaker 4: here's the one for the client, here's the one for
Speaker 4: the website, et cetera, et cetera. Imagine the creation of
Speaker 4: pitch books for clients where you take ten plates, you
Speaker 4: put a bunch of data, you go out and do research,
Speaker 4: you take logos, you take this, you take that. There
Speaker 4: is a lot of that machinery and factory, which you know,
Speaker 4: we have thousands of people doing that I'm sure there's a.
Speaker 1: Lot of junior analyst who would be maybe glad to
Speaker 1: hear that some of making a pitchbox is going to
Speaker 1: be on.
Speaker 4: But I think that's a good thing. It takes away
Speaker 4: some of the toil. And so I think at the
Speaker 4: end of the day, listen right now, have you noticed
Speaker 4: that everything is kind of converging to words and concepts,
Speaker 4: no matter if you're a developer, if you're a knowledge worker,
Speaker 4: those jobs are candle colliding. And I'm absolutely developers have
Speaker 4: seen that first. Why well, because it's a low hanging fruit.
Speaker 4: The developers deal with the vocabulary. There is no fifty
Speaker 4: thousand words. There's like two three hundred keywords for language,
Speaker 4: and so of course that works extremely well, and of
Speaker 4: course that's the first thing to go. But I think
Speaker 4: eventually the knowledge worker is going to be, you know,
Speaker 4: the one that is really benefit and no matter if
Speaker 4: you are a developer or or or if you are
Speaker 4: working on a pitch book, or if you're working on
Speaker 4: a summarization of a meeting or the action items, or
Speaker 4: you're working on a strategy, et cetera, et cetera. And
Speaker 4: I think overall this will elevate the quality of the work,
Speaker 4: which then everybody says a happy worker or a happy
Speaker 4: developer is a productive developer. I think you're happy when
Speaker 4: you're actually doing something that allows you to do your
Speaker 4: best work. And I'm hoping that if AI allows all
Speaker 4: of us to do more of our best work, I
Speaker 4: think it's going to be, you know, probably the biggest
Speaker 4: effect that we can have.
Speaker 1: I know, we just have a couple more minutes. So
Speaker 1: one very quick question, what makes a good prompt?
Speaker 4: Well, believe it or not. Empathy. You need to be empathic,
Speaker 4: and you need to be gentle, and you need to
Speaker 4: be kind, and you need to kind of, you know, just.
Speaker 1: Like empathetic.
Speaker 2: She makes fun of me for how empathement.
Speaker 1: You know, I've said, it's very sweet that you say.
Speaker 4: You need to take the AI literally by the hand
Speaker 4: and take it where you want to go. And I
Speaker 4: tell you that, you know, one of my interesting, more
Speaker 4: interesting experience with prompts is the following. You know how
Speaker 4: hard it is to get an AI to say I
Speaker 4: don't know. It's almost impossible. You're always going to get
Speaker 4: an ass And so one time I decided I want
Speaker 4: to get it to the point, and so I had
Speaker 4: to navigate the prompt and the AI to understand that
Speaker 4: it was safe and okay to say I don't know.
Speaker 4: And so then at the end I prompted it, what's
Speaker 4: the capital of you know, the United States? Okay? And
Speaker 4: then I said that you know, what's the weather going
Speaker 4: to be like tomorrow? And I got an answer, and
Speaker 4: then I said what's the weather going to be in
Speaker 4: a year, and it's simply I don't know. And then
Speaker 4: at one point, you know, I even decided what to say.
Speaker 4: It's like, is there a role for humans in a
Speaker 4: world of a eye?
Speaker 2: I don't want to know?
Speaker 1: Okay, Well, everyone's going to be off on chat GPT
Speaker 1: now trying to get it to say I don't know.
Speaker 1: Marco Argenti from Goldman Sachs, thank you so much. That
Speaker 1: was good fun.
Speaker 2: Thank you, Johing, thank you so much.
Speaker 1: Thank you so much, Joe. That was a lot of fun.
Speaker 1: And I have to say I do not make fun
Speaker 1: of you for saying please and thank you to Chat GPT.
Speaker 1: I have I'm going to repeat it. I've said it's endearing,
Speaker 1: it's very sweet, and I've tried to follow your example.
Speaker 1: And I now I don't say thank you because I
Speaker 1: usually move on to the next question. But I do
Speaker 1: say please.
Speaker 2: I've heard this though.
Speaker 3: It's funny that you said that, because I actually have
Speaker 3: heard this that there does seem to be quantitative evidence
Speaker 3: that words like please and thank you, et cetera do
Speaker 3: actually improve really well, yeah, mad Buseegan, who you know
Speaker 3: we've known on Twitter forever, has posted about this. So
Speaker 3: there's a good reason to do it besides just the
Speaker 3: habit the all entities you talk to, you should be
Speaker 3: in the habit of flight.
Speaker 1: Oh yeah, that was your argument, right, yeah, yeah, yeah, Okay,
Speaker 1: Well I thought that was fascinating.
Speaker 2: Yeah.
Speaker 1: We've been talking a lot about AI and the sort
Speaker 1: of potential use cases and the chips that are driving
Speaker 1: the technology and things like that, but it was nice
Speaker 1: to hear from someone who's actually making the purchasing decisions, yes,
Speaker 1: and implementing them at a large institution.
Speaker 2: Absolutely.
Speaker 3: That was probably one of my favorite AI conversations we
Speaker 3: had for precisely that reason, because it was interesting hearing
Speaker 3: him talk about this idea that right now, like these
Speaker 3: open source models, particularly like the latest version of LAMA,
Speaker 3: is getting really close to sort of the core proprietary models.
Speaker 3: That was striking the fact that he sees, perhaps particularly
Speaker 3: on the inference side of model usage, an opportunity for
Speaker 3: greater use of different types of hardware.
Speaker 1: Also very interesting, that's right, And we're so used to
Speaker 1: talking about the massive amounts of power and energy that
Speaker 1: AI will consume, and we you and I have had
Speaker 1: a lot of conversations about how we're going to power
Speaker 1: all these servers and things. But what's gotten far less
Speaker 1: attention is just optimizing the way you use AI such
Speaker 1: that you don't need to consume as much power, So
Speaker 1: maybe doing less training, leaving training to the big like
Speaker 1: hyperscalers or whatever, and then just doing the inference.
Speaker 3: In the end, it's going to be both, right, because
Speaker 3: in the end, like there's both, it's going to happen.
Speaker 3: People are gonna find algorithmic techniques and Marco described some
Speaker 3: of them to lessen the sort of pressure and stress
Speaker 3: that you're putting on the hardware, but of course that's
Speaker 3: just going to mean you're going to use it more.
Speaker 3: And then also people are going to have to solve
Speaker 3: the power consumptions. That kind of like all of economic
Speaker 3: history in general, in which we're always finding new ways
Speaker 3: to get more out of the same you know, gigajewel
Speaker 3: of energy but also using more energy at the same time.
Speaker 4: Yeah.
Speaker 1: Absolutely, well, shall we leave it there.
Speaker 2: Let's leave it there.
Speaker 1: This has been another episode of the aud Thoughts podcast.
Speaker 1: I'm Tracy Alloway. You can follow me at Tracy Alloway.
Speaker 3: And I'm Jill Wisenthal. You can follow me at the Stalwart.
Speaker 3: Follow our producers Carman Rodriguez at Carman Ermann dash O,
Speaker 3: Bennett at Dashbot, and kel Brooks at Kelbrooks. Thank you
Speaker 3: to our producer Moses ONEm and from our Odd Lots content.
Speaker 3: Go to Bloomberg dot com slash od loss. We have transcripts,
Speaker 3: a blog, and a newsletter, and you can chat about
Speaker 3: all of these topics in our discord where we even
Speaker 3: have an AI channel. Great stuff in their discord dot
Speaker 3: gg slash Odlins.
Speaker 1: And if you enjoy Odd Loots, if you like our
Speaker 1: continuing series of AI conversations, then please leave us a
Speaker 1: positive review on your favorite podcast platform. And remember, if
Speaker 1: you are a Bloomberg subscriber, you can listen to all
Speaker 1: of our episodes absolutely ad free. All you need to
Speaker 1: do is connect your Bloomberg account with Apple Podcasts. In
Speaker 1: order to do that, just find the Bloomberg channel on
Speaker 1: Apple Podcasts and follow the instructions there. Thanks for listening.