Speaker 1: Bloomberg Audio Studios, Podcasts, Radio News. Welcome to Marin Talks Money,
Speaker 1: the podcast in which people who know the markets explain
Speaker 1: the markets. I am Maren Summerzet Web and this week
Speaker 1: I am speaking with doctor Yanushmeretzky, who is an AI
Speaker 1: partner at Aaron Innovation Capital. Now, as you know, on
Speaker 1: this podcast, we like to talk about the big forces
Speaker 1: affecting our economy and markets in general, so it's really
Speaker 1: no surprise that we keep coming back to the impact
Speaker 1: of AI. We've talked at different times about the consequences
Speaker 1: for jobs, for inflation, for interest rates, for tech companies,
Speaker 1: and whether politicians and indeed policymakers, let alone ordinary workers
Speaker 1: and investors, are ready for any of this. But what
Speaker 1: we've never asked is is it actually working? Yanish Welcome
Speaker 1: to Marri Talks Money.
Speaker 2: Thank you so much for having me.
Speaker 1: And it's coming to start with a brief explanation of
Speaker 1: exactly what it is that we mean when we say AI.
Speaker 2: Yes, everyone talks.
Speaker 1: About AI all the time. Aire say that is going
Speaker 1: to change the world, it's going to solve all our problems,
Speaker 1: it's going to destroy our jobs, etc. But what do
Speaker 1: we actually mean when we say AI.
Speaker 2: So these days, what we mean by saying AI is
Speaker 2: a system which is approximating certain process. It might be
Speaker 2: a system which is approximating language. It might be a
Speaker 2: system which is approximating images. It might be a system
Speaker 2: which is approximating how a robot moves. By the end
Speaker 2: of the day, the current generation of AI techniques, those
Speaker 2: neural networks are the function approximators. This is approximate things.
Speaker 2: They don't solve intelligence, they approximate. So that's why you
Speaker 2: may have an illusion that those systems are intelligent. At
Speaker 2: the end of the day, they are approximating intelligence.
Speaker 1: And I suppose there's two ways to look at this.
Speaker 1: What AI as you've just described needs from us and
Speaker 1: what we really need from it to make it work
Speaker 1: for us. So if you look at the way that
Speaker 1: the big hyperscalas are approaching things at the moment, they're
Speaker 1: building massive data centers to build out their capacity, and
Speaker 1: that requires vast amounts of energy, It requires lots of coolants,
Speaker 1: it requires a very large volume of very different types
Speaker 1: of chips, right, yes, and all those things. Obviously there
Speaker 1: are troubles at the moment. We're getting all these things
Speaker 1: with the war in the Middle East, etc. So there
Speaker 1: are supply restrictions, but nonetheless none of these things are
Speaker 1: really our long term problem. Given their correct policy choices,
Speaker 1: all those material elements can easily be not easily, but
Speaker 1: can be found and built in. Then there's the second
Speaker 1: bit that we've talked about when we met we last met,
Speaker 1: which is the data that you require to train a model,
Speaker 1: and that can't be created in the volumes that are required.
Speaker 1: And we've hit it. We've hit a supply problem with data.
Speaker 2: We have hit a supply problem with diverse data. To
Speaker 2: clarify the thing, because you can just create an infinite
Speaker 2: amount of data by randomly generating new words, so you
Speaker 2: can create it. But we're talking about creating high quality,
Speaker 2: diverse data. And you see we've run out of data,
Speaker 2: diverse data, not yesterday, not a month ago. We have
Speaker 2: run out of data three and a half years ago.
Speaker 2: You can to understand that the last frontier model GBT four,
Speaker 2: which was not a combination of agents, etc. Just basic LLM.
Speaker 2: The last LLM that was the most potent was GBT four.
Speaker 2: It was released when it was released in twenty twenty
Speaker 2: three in January, However, the training of that model finished
Speaker 2: at the end of twenty twenty two, so it's three
Speaker 2: and a half years ago. We have trained the model
Speaker 2: that used all the publicly available data on the Internet.
Speaker 2: There is nothing more out there to use to train
Speaker 2: the model. Hit the data ceiling not a month ago,
Speaker 2: but three and a half years ago. It's extremely important.
Speaker 2: And yes, what we're doing right now is we're trying
Speaker 2: to put together multiple lms. We're trying to have synthetic data,
Speaker 2: but the performance isn't really there. We've hit diminishing returns
Speaker 2: not a month ago, but three and a half years ago.
Speaker 1: Okay, And it's surely new data is created all the
Speaker 1: time on the Internet. You know, we talk about diverse data,
Speaker 1: and it's an awful lot more created in the past
Speaker 1: than there is now. But everyone's use of the Internet
Speaker 1: surely creates huge volumes of new data all the time.
Speaker 2: That's even worse. Humans are creating new data on the Internet,
Speaker 2: but that data falls into certain patterns. How many conversations
Speaker 2: on the weather you can have every day?
Speaker 1: A lot, actually a lot. I think I've already had
Speaker 1: three today, to be honest. You know, it's very cold
Speaker 1: in Edinburgh.
Speaker 2: You can have a lot, but there's a bigger, profound
Speaker 2: problem there I want to mention is that if you
Speaker 2: look at the data that is currently being created on
Speaker 2: the open Internet, it is data create by those llms,
Speaker 2: which number one is inaccurate because it suffers from Halla's hallucinations,
Speaker 2: and number two, it feeds into itself, so that the
Speaker 2: new models now train on the entire Internet, the training
Speaker 2: on the output from other models, which is I said,
Speaker 2: are making mistakes and hallucinations, and that, using a technical term,
Speaker 2: slowly ends up leading to something called model collapse, where
Speaker 2: the models themselves are actually getting dumber.
Speaker 1: Okay, so if you train a new model on newly
Speaker 1: created data, you're effectively training on its own nonsense or
Speaker 1: nonsense created by other similar models.
Speaker 2: Yeah, well, not only nonsense. We have to understand that
Speaker 2: those llms using artificial neural networks do you're correct let's
Speaker 2: say ninety five ninety nine percent of the time, So
Speaker 2: most of the content is correct, but you no longer
Speaker 2: know what content is incorrect, which is a significant degradation
Speaker 2: of the quality of data. It's almost like having access
Speaker 2: to a calculator which claims to be correct one hundred
Speaker 2: percent of the time, but in reality it is correct
Speaker 2: ninety five ninety nine percent of the time. How much
Speaker 2: would you pay for that calculator? Would you use the
Speaker 2: output from that calculator to produce novel calculator? No, you
Speaker 2: wouldn't do that. But we are doing it right now,
Speaker 2: and we are seeing right now that the performance of
Speaker 2: those models on benchmark is not increasing anymore plat out
Speaker 2: three years ago. And there's even a bigger, profound question.
Speaker 2: We are training those models now using higher quality data,
Speaker 2: so no longer we're using the entire internet. Now we
Speaker 2: are filtering from the Internet a lot of garbage, so
Speaker 2: we're using a smaller training data set, thus producing smaller
Speaker 2: llms that have the same quality. But as I mentioned,
Speaker 2: they are smaller, So what is the consequence of it. Well,
Speaker 2: as we speak, I'm running those models on my laptop.
Speaker 2: I don't use any data center for it. I never will.
Speaker 2: I'm running them on my laptop. So the models have
Speaker 2: gotten better. But by saying better, I mean they are
Speaker 2: more power efficial.
Speaker 1: They're not all right, So I mean that they're most specific,
Speaker 1: so that I get toward a more specific task because
Speaker 1: they're using a narrower range of data, so they can't
Speaker 1: be a type of general intelligence they're specific and.
Speaker 2: Remarkably no, they are general purpose. I mean you can
Speaker 2: just go online and down on quent three point five
Speaker 2: and three point six. This is a general purpose model,
Speaker 2: which is even accessing the internet to give you Summari
Speaker 2: is reasoning. It's doing this thing on your laptop. It's today.
Speaker 2: Of course. On top of that, if you want to,
Speaker 2: you can find tune your model on your proprietary data.
Speaker 2: But you see even the fine tuning process these days
Speaker 2: can be done on your laptop, which again raises the question,
Speaker 2: why do you need to pay for compute? Why do
Speaker 2: you need all this expansion of the data centers. You
Speaker 2: really don't. And what's going to happen in year two,
Speaker 2: three years from now when the majority of laptops out
Speaker 2: there will be able to run general purpose the most
Speaker 2: potent language models with access to the Internet. I just
Speaker 2: I'm just not seeing it. And that's on top of
Speaker 2: one fundamental thing that I want to mention upfront, because
Speaker 2: you mentioned what does the AI need from us? And
Speaker 2: what we need from the AI? So what the AI
Speaker 2: needs from us are two things, as you mentioned. Number one,
Speaker 2: compute and yes, we can provide more and more compute
Speaker 2: as long as oracle CDs is not going down, which
Speaker 2: it is right now. And number two, we need to
Speaker 2: provide more data. We've use all of it, so to
Speaker 2: some extent we cannot produce better AI. Now what do
Speaker 2: we want to have from the AI. We want to
Speaker 2: have at least two things and just bury me here.
Speaker 2: Number one, we want to have systems which are continually learning.
Speaker 2: So just like during this podcast today, I hope that
Speaker 2: I will be able to learn something from you. You
Speaker 2: will be able to learn something from me and remember
Speaker 2: tomorrow or maybe remembered after Easter. Current generation systems in general,
Speaker 2: neural networks are not learning anything new you when you
Speaker 2: interact with them, which is significant limitation. So we're not
Speaker 2: getting from the AIS what we want to They're not
Speaker 2: learning from us. It's a fundamental limitation that's not solved.
Speaker 2: And number two, you want to mention this thing up front.
Speaker 2: Those systems are stochastic. They are probabilistic. You cannot trust them.
Speaker 2: They roll the dice whenever they produce output, so to
Speaker 2: some extent you cannot trust their output. Can you make
Speaker 2: them deterministic? Yes, of course you can by making sure
Speaker 2: that they always produce the most likely token. Now, use
Speaker 2: the word token. But the problem with this thing is
Speaker 2: that then they will be just copying the data from
Speaker 2: the training set. So just imagine those lawsuits when you
Speaker 2: see verbatting copies of all the podcasts books produced as
Speaker 2: an output of Gemini or open Ai system. So those
Speaker 2: systems number one are not continually learning, which basically for
Speaker 2: me is just a no go. And number two, those
Speaker 2: systems are not to be trusted because whenever they produce output,
Speaker 2: that output is it stochastic. It's not a terministic it's
Speaker 2: basically rolling the dice to catch you output.
Speaker 1: Yeah. Can we talk a bit more about that about
Speaker 1: how the hallucinations or errors have it build up?
Speaker 2: Absolutely? I me try to explain using simple terms when
Speaker 2: you use ch GPT or Gemini or copilot, but this
Speaker 2: is Copilot is actually open eye system. It just produces texts,
Speaker 2: so you have an impression that generates one word at
Speaker 2: the time deterministically, that's what's on the screen, just one
Speaker 2: word after another. But in reality that's not what those
Speaker 2: systems produce. If you are a developer like me and
Speaker 2: my coelagus are developers as well, you can look at
Speaker 2: the developer output of a large language. Models do you
Speaker 2: know what it gives you. It gives you a vector
Speaker 2: of fifty thousand elements, actually fifty two thousand elements, so
Speaker 2: fifty thousand elements, and each element has a certain probability
Speaker 2: of being correct zero point ninety five zero one zero
Speaker 2: zero one zero, zero point three. It's an entire vector.
Speaker 2: It's not one word has probability one. Everything else is zero. No,
Speaker 2: there is a little bit of error there. It has
Speaker 2: to be, and so not is what happens when you're
Speaker 2: producing one word at the time, or one token at
Speaker 2: the time. You can think of a token as a
Speaker 2: ward split by two or three. So when you produce
Speaker 2: output one token at the time, this system is rolling
Speaker 2: the dice all the time. It's making a small, tiny
Speaker 2: error every time it produces a ward s. At the beginning,
Speaker 2: you may not perceive the error, but over time, after
Speaker 2: three hundred, five hundred a thousand words, the error is
Speaker 2: going to be so big that it's going to result
Speaker 2: in the critical failure of the system. You cannot circumvent
Speaker 2: it because those systems are probabilistic. It's not like in
Speaker 2: an Excel spreadsheet, where you can have a chain of
Speaker 2: aund ten twenty one hundred one thousand formulas and you
Speaker 2: know that the formulas are going to produce the correct result. Here,
Speaker 2: if you have a chain of words, every time you
Speaker 2: produce a word, you accrue a little bit of error
Speaker 2: and you see this error manifest itself later on.
Speaker 1: Okay, so is this solvable? I guess that's the key question.
Speaker 1: Is it possible? And the type of models that we're
Speaker 1: using at the moment, which are super hyped, is it
Speaker 1: possible for that hallucination problem or compounding error problem to
Speaker 1: be solved, or is it simply a systemic shortcoming that
Speaker 1: is unresolvable.
Speaker 2: So here I'm again speaking from my experience having two
Speaker 2: ephds in mathematics and computer science and twenty years of
Speaker 2: experience in deep mind and IBM Watson research. No, it's impossible.
Speaker 2: You have to use a different technique for it. Yes,
Speaker 2: there are different techniques that are emerging right now. In
Speaker 2: full disclaimer, I'm also a co founder of a startup
Speaker 2: working on one of those techniques. We call them fractal brain. Yes,
Speaker 2: there are new techniques on the horizon. However, the existing
Speaker 2: techniques have a building mechanism so that every time produce
Speaker 2: an output another word, you have a little bit of error.
Speaker 2: You cannot eliminate it thing. And there's also a fundamental
Speaker 2: other thing you mentioned hallucinations. What do we mean by hallucinations?
Speaker 2: It's number one, it's producing those small errors one word
Speaker 2: at a time. But there's also another reason for hallucinations
Speaker 2: when this system just doesn't remember what was mentioned yesterday,
Speaker 2: a week ago, or a month ago, and goes back
Speaker 2: to the initial question. You need both. You need to
Speaker 2: have a system which, like humans, it's continually learning. And
Speaker 2: number two, you need to have a system which does
Speaker 2: not make errors when it produces the next word. It
Speaker 2: shouldn't roll a dice. You need those two and so
Speaker 2: now to answer your question, can we solve this problem
Speaker 2: using artificial neural networks? No, there are attempts to. There
Speaker 2: have been attempts to circumvent it. If you want to
Speaker 2: have continual learning, you can maybe try to use something
Speaker 2: like continual backpropagation from rich Sutt only got a two
Speaker 2: in go word in twenty twenty four. Or we can,
Speaker 2: but it's not solving the problem. At least he's attempting
Speaker 2: to find solutions to it. You can retrain the system,
Speaker 2: of course, right, I mean, why not take GBT for
Speaker 2: after the end of the podcast. Retrain the entire system.
Speaker 2: It's going to cost you five million dollars. But you
Speaker 2: could do that. You could retrain the entire system, or
Speaker 2: you can find you in your system. But when you
Speaker 2: find you in your system, the system is forgetting what
Speaker 2: it was trained on before. So you're suffering something which
Speaker 2: is called catastrophic forgetting or catastrophic interference. Long story short, No,
Speaker 2: you cannot solve the outstanding problems of hallucinations and lack
Speaker 2: of continual learning. Unfortunately. I wish you could, but you
Speaker 2: need to have different things for it. And it's not
Speaker 2: just me saying it. Look at the landscape of researchers,
Speaker 2: leading researchers in the field, my colleagues, I know all
Speaker 2: of them personally. They have all jumped ship. This is important.
Speaker 2: Look at, for example, young Lecon from Metaga. He jumped ship.
Speaker 2: He's working on his new startup am I line apps,
Speaker 2: not working on llms, saying l lams is a dead
Speaker 2: end to AGI. Look at Michael Iff Dave Silver from DeepMind.
Speaker 2: He just left deep Mind, but a couple of weeks
Speaker 2: ago he formed Ineffable Intelligence. I think they're raising a
Speaker 2: billion dollars the same thing. He is not a believer
Speaker 2: of using l lms for general artificial intelligence, and can
Speaker 2: go on and on Iliasutskiver, Andre Carpaty. And so it's
Speaker 2: not just me who is saying that we need to
Speaker 2: go back to research. The leading research and researchers in
Speaker 2: the field have already jumped ship a few months, maybe
Speaker 2: even years ago, working on the next generation things. The
Speaker 2: market still believes we can solve hallucinations, but the leading
Speaker 2: researchers have jumped ship. That's unbelievable to me that we
Speaker 2: keep pouring money into bigger data centers, knowing we've used
Speaker 2: all the data already, and knowing that even if they
Speaker 2: have more data, you will not solve continual learning, and
Speaker 2: you will not solve hallucinations. So why is not everyone?
Speaker 2: Why are people doing it?
Speaker 1: Okay, So if we know clearly, and it sounds from
Speaker 1: what you say that we do know very clearly that
Speaker 1: using more and more computing power and more and more
Speaker 1: already mildly corrupted data isn't going to get us anywhere.
Speaker 1: Then the enormous cap expand on vast data centers, the
Speaker 1: hundreds of billions of dollars that have already gone into
Speaker 1: this and are still projected to go into This is
Speaker 1: a catastrophic misallocation of capital.
Speaker 2: Well that's how you look at it. So obviously you
Speaker 2: can say the biggest winner of it is Nvidia because
Speaker 2: it's producing those GPUs or during the gold rush, you
Speaker 2: should invest in companies that produce the shovels you can
Speaker 2: still make money. For example, I'm a partner at Urn
Speaker 2: Innovation Capital in the UK and we are investing in
Speaker 2: airs mission two companies, one called high Verge and second
Speaker 2: called Hydra. These companies improve coolings of data centers or
Speaker 2: these companies produce better algorithms to run on data centers.
Speaker 2: So you can still allocate your capital wisely, but you
Speaker 2: shouldn't allocate them in companies which are spending on this compute.
Speaker 2: You should allocate your capital in companies that are allowing
Speaker 2: those data centers to run efficiently because those data centers,
Speaker 2: who knows, maybe they will be used for different purpose
Speaker 2: at some point. So again, there are going to be
Speaker 2: winners and losers of the current gold rush in GPUs
Speaker 2: and AI. I would say companies that have not invested
Speaker 2: massively in data centers and in front here models are
Speaker 2: going to be the winners. There's some companies without mentioning
Speaker 2: the names, that have been accused of not training their
Speaker 2: own language models. I think these, to me are going
Speaker 2: to be the winners in today's market.
Speaker 1: Are we talking about Apple?
Speaker 2: I will let you determine that thing. But you can
Speaker 2: see some companies number one, have not invested in the
Speaker 2: LM front here models, but the research teams have kept
Speaker 2: public papers saying that those lms stayed on the reason
Speaker 2: they make mistakes. So let you find those companies. And
Speaker 2: there are some companies that have borrowed a lot of
Speaker 2: money to expand those center centers. And it's not me.
Speaker 2: Look at the market, look at CDs on Oracle. It's
Speaker 2: the market is just flashing reds saying this is foolish.
Speaker 2: So we're seeing those signals already. But I want to
Speaker 2: make sure that if you want to make money in
Speaker 2: today's market, is given that the governments have to pay
Speaker 2: eight nine percent of the revenue on servicing the debt.
Speaker 2: You don't know if the market is going to go
Speaker 2: up and down, and maybe a capital injection, liquid, the
Speaker 2: injection from Central Bucks. You don't know those things. So
Speaker 2: I would not recommend your short or go along any investment,
Speaker 2: I would recommend maybe doing an arbitrush buy companies that
Speaker 2: have not wasted money on llms and short companies that
Speaker 2: have borrowed a lot of money to expand data centers.
Speaker 2: That will be my suggestion, but I might be wrong.
Speaker 1: Again. Can you tell us about any of the startup
Speaker 1: companies that you're invested in or interested in that are
Speaker 1: taking us to this new frontier and AI away from
Speaker 1: the LLM model and towards a different model.
Speaker 2: Absolutely so. In our investment fund, our an innovation capital,
Speaker 2: we have access to a lot of companies, actually maybe
Speaker 2: twenty or thirty companies that are developing the next frontier
Speaker 2: models which are not necessarily using artificial in our networks.
Speaker 2: And I can speak about three or four of them
Speaker 2: which are very exciting. One company you can have a look.
Speaker 2: They are based in Switzerland. They're called Innate AI. Again,
Speaker 2: I'm advertising them and we're not investing in them. We're
Speaker 2: not investing them yet. They are developing new version of
Speaker 2: neural networks which are inspired on the brain. This is
Speaker 2: an effort that was going on in Europe for more
Speaker 2: than a decade, the Blue Brain project. They are developing
Speaker 2: something new which is not an artificial neural network. So
Speaker 2: that's one kind. Look at another company, for example, PATHWAYAI
Speaker 2: in the Bay Area. Again another example they mentioned upfront,
Speaker 2: you need to solve continual learning. You need to solve that.
Speaker 2: If you don't have it, forget about the solution to AGI.
Speaker 2: And so they have been developing systems that can learn
Speaker 2: using something called heavy on learning, which is a local
Speaker 2: learning technique that happens in the brain, not using bad
Speaker 2: propagation grid in descent. So this is another example. Another
Speaker 2: company that I'm actually a co founder of and the
Speaker 2: CEO called Fractal Brain AI. Have a look at the thing.
Speaker 2: It's also based on prefrontal cortex, and it's this idea
Speaker 2: that those networks are continually growing and rewiring themselves. So
Speaker 2: no longer you have a fixed network with a fixed
Speaker 2: number of parameters. No, the network is growing expanding themselves.
Speaker 2: Like today, you're going to probably form a connections after
Speaker 2: this podcast. Those networks do the same, They create new
Speaker 2: connections all the time, and on top of that, they
Speaker 2: are continually learning and thousands of times more power efficient
Speaker 2: in addition to being data efficient. So these are only
Speaker 2: some of the examples of companies that I'm personally very
Speaker 2: excited about. But as ILIOSU Skiverer said the other day
Speaker 2: on one of those podcasts, that we have gone back
Speaker 2: from the age of scaling to the age of research,
Speaker 2: So researchers have gone back to developing the new things.
Speaker 2: It's just onlin end here me and my teams have
Speaker 2: started developing, for example, fractals fractal brain twelve years ago.
Speaker 2: We knew about those outstanding limitations of artificial neural networks
Speaker 2: more than a decade ago, so we wouldn't invest our
Speaker 2: time in it.
Speaker 1: Nevertheless, somehow I got to cut up in this sort
Speaker 1: of super bubble hype, despite the fact that good scientists
Speaker 1: aren't you.
Speaker 2: But this is good. I like the hype because you see,
Speaker 2: to some extent that hype and the lands they allowed
Speaker 2: us to understand that it's possible to approximate human language.
Speaker 2: So now that you know that you can approximate human
Speaker 2: language with el lamps, you can try to find ways
Speaker 2: to actually solve the idea of human language. If you
Speaker 2: can approximate something, you can see the size of it.
Speaker 2: So it's almost like if someone showed you, hey, there
Speaker 2: is a rocket there, it flies, you already know the
Speaker 2: size of the rocket. You can know you can fly,
Speaker 2: you can start to crack the details of the engine
Speaker 2: of the rocket. So to some extent, I like the
Speaker 2: current hype. I like the current generation systems because they
Speaker 2: allowed us to understand the size of the problem and
Speaker 2: approximate it. Now that we know that we can in
Speaker 2: principle approximate human language, let's just solve it, okay.
Speaker 1: And the other thing I suppose we should say is
Speaker 1: that while we spent quite a lot of time criticizing
Speaker 1: this generation of llms, they're still great, still as really useful.
Speaker 1: It's not like we have a totally pointless technology, something
Speaker 1: that can remove entry level jobs across the board, which
Speaker 1: of course comes with its own problems, but none LASS
Speaker 1: has enormous use from activity and business.
Speaker 2: Absolutely, I love maybe not just l elms. I love
Speaker 2: the generative AI. For example. Not sure if you've noticed
Speaker 2: behind me, I have this amazing landscape of London, but
Speaker 2: you can tell it's all fake here, like this building
Speaker 2: is all tilted here. So those systems they produce very
Speaker 2: pretty graphics. It's inaccurate, but it's okay for me. It
Speaker 2: still gives me a very nice background. Same thing with text.
Speaker 2: They will produce beautifully looking poem. They can summarize the document. Yes,
Speaker 2: there are errors. The are like this building here. You
Speaker 2: can tell it's all tilted a little bit, But I'm
Speaker 2: okay with that. So those systems are very good for
Speaker 2: creating templates, knowing what templates of data, it's so called
Speaker 2: boiler plate code, creating nice graphics. They're not good for
Speaker 2: details to put in there. And it's interesting because about
Speaker 2: two or three years ago, I was giving a talk
Speaker 2: to high school students. They're asking me what do we
Speaker 2: use those lms for. I told them for generating templates,
Speaker 2: templates of presentation, etc. And for summarizing documents. But I
Speaker 2: misled them. I don't think you should be using those
Speaker 2: systems for summarization for two reasons. The reason number one,
Speaker 2: in that summary there might be errors and mistakes, So
Speaker 2: if you summarize a document, don't throw away the original.
Speaker 2: And number two, you know, when you're summarizing something, you
Speaker 2: should know what you care about. For example, if you
Speaker 2: are to summarize today's podcast, maybe you only care about
Speaker 2: this tilted building here, which is fake. Maybe that's what
Speaker 2: you're looking for. LM doesn't know it. So when it
Speaker 2: produces a summary of text. When it compresses graphics, it
Speaker 2: doesn't know what you care about really, so it's going
Speaker 2: to produce a summary maybe lacking the details that you
Speaker 2: want to know later on.
Speaker 1: And I suppose the other things with the errors. You
Speaker 1: should only really be using it for things where you
Speaker 1: know that you will be able to spot the errors
Speaker 1: at the end.
Speaker 2: So this is a actually very interesting I think that
Speaker 2: the killer use case for generative AI is producing output
Speaker 2: that you yourself can check for correctness. So for example,
Speaker 2: if I'm if I want to compute one hundred plus
Speaker 2: one hundred, LM will give me an answer I know
Speaker 2: I can check the answer myself the correctness. I like
Speaker 2: it this way. Unfortunately, people are using those lms to
Speaker 2: answer a question to which they themselves don't know the
Speaker 2: answer to. This is a recipe of an absolute disaster.
Speaker 2: In the worst case, you can use those systems to
Speaker 2: check whether your output that you produce yourself is correct.
Speaker 2: You can do it that way. But people using the
Speaker 2: other way around. They are asking those lms to produce
Speaker 2: an output they don't know what the output should be,
Speaker 2: and the output can have one or five percent error, right,
Speaker 2: why would you even do that?
Speaker 1: So are we worrying unnecessarily about the job market. We
Speaker 1: spend most of the talks and podcasts and panels, etc.
Speaker 1: That I do, the question is always what on earth
Speaker 1: does my child do for work? In an age of AI.
Speaker 1: Are we worrying too much about that? Because the human
Speaker 1: input will remain absolutely compulsory for the next couple of decades.
Speaker 2: Having the same problem as I mentioned, my son as
Speaker 2: is thirteen years old, so mis the father to give
Speaker 2: him advice what to do in the future. I guess
Speaker 2: being a scuba diver instructor is a good job. Yeah,
Speaker 2: it is a great job. You're going to need them.
Speaker 2: I worry about interns entry level of software engineers because
Speaker 2: to some extent you can automate. It's not replaced. You
Speaker 2: can automate most of the tasks that you delegate to
Speaker 2: interns today, like, for example, write a boilerplate code, template code,
Speaker 2: check some if there's errors in that code, you can
Speaker 2: automate that. But what's going to happen then is that
Speaker 2: we are not going to have interns anymore or significantly
Speaker 2: smaller number of interests and entry level software engineers. So
Speaker 2: what's the consequence. What's going to happen with the entire
Speaker 2: promotion cycle. If you're in senior management or a middle management,
Speaker 2: you're gonna get promoted. Who's going to replace you? Who's
Speaker 2: going to become a software architect if we're not training
Speaker 2: the new entry level software engineers. So there's this entire
Speaker 2: skills gap right now. Some people that I know have
Speaker 2: chosen not to pursue studies in computer science because for
Speaker 2: the very reason they worry that we're not going to
Speaker 2: need software engineers. Yes, we are going to need software engineers,
Speaker 2: it's just you need to jump immediately to being an
Speaker 2: architect of a software engineering system. And to make sure
Speaker 2: verry hard. It is hard. It Typically what you do
Speaker 2: is you get gain this experience on the job. You
Speaker 2: go to Google, spend first two three years writing code,
Speaker 2: but you appear so software architects, and so you learn
Speaker 2: from them. If we don't have this experience that we're
Speaker 2: giving to entry level software engineers, they won't be able
Speaker 2: to have this experience. So this is what always me more.
Speaker 2: It's not about replacing software engineers, it's about us not
Speaker 2: having a pipeline of software architects. And see your software engineers,
Speaker 2: we are not having this pielime anymore. That worries me
Speaker 2: quite a bit, to be honest.
Speaker 1: Yeah, and the pipeline problem is just being discussed in
Speaker 1: a lot of other professions as well, most obviously in
Speaker 1: a legal profession.
Speaker 2: Absolutely, again for lawyers, this is very interesting and this
Speaker 2: is not my profession, so you can discount what I
Speaker 2: say quite a bit way. Now, you can get the
Speaker 2: initial blueprints of legal documents very quickly, and we do
Speaker 2: it all the time at our startup. You can get
Speaker 2: a blueprint of a legal document, but I would never
Speaker 2: use that document to get an investor on board in
Speaker 2: my startup. I wouldn't do that. I still need to
Speaker 2: send it to an actual human being to at least
Speaker 2: prove read it. So we're going to have to have
Speaker 2: those lawyers which can prove readdocuments produced by the generative AI,
Speaker 2: but aren't doing it already. They already have hundreds of
Speaker 2: template documents safe on their computers. It's just changing the
Speaker 2: names of companies investors in those documents. So the legal
Speaker 2: profession is not going to go away because those generative
Speaker 2: AI systems, they don't have the notion of true and false.
Speaker 2: They don't they confused. It's all probabilistic, so they will
Speaker 2: make an error. Sometimes after twenty thirty forty legal statements,
Speaker 2: they will just change one true into false. So we're
Speaker 2: still going to need to have lawyers for it. This
Speaker 2: is one of those professions where I don't think it's
Speaker 2: going to be automated fully by AI. But as I said,
Speaker 2: there are other professions that have already been automated. For example,
Speaker 2: content creators. So you can go online, go to YouTube.
Speaker 2: You're going to see a lot of videos summarizing AI,
Speaker 2: AI bubble summarizing conflict in Ukraine or Iran, all generated
Speaker 2: using generative AI. So some jobs have already gone away,
Speaker 2: the jobs that require to be you to be one
Speaker 2: hundred percent accurate, those jobs are not going to go
Speaker 2: away using the current generation of AI systems. Remember, there
Speaker 2: are new generations on the horizons. My startup is working
Speaker 2: on it, other startups are working on it. But with
Speaker 2: the current generation of tools, they will not displace those
Speaker 2: jobs yet.
Speaker 1: Yeah, it sounds to me like we shouldn't necessarily be
Speaker 1: frightened the current generation of tools, but we should hardly
Speaker 1: be pretty frightened the next generation.
Speaker 2: I would give a very simple example here. So me
Speaker 2: and my colleagues we've built Alpha go at Deepmine in
Speaker 2: twenty fifteen, which is the system that won in computer
Speaker 2: Goal with the World champion, and so back then it
Speaker 2: was a state of the art system. These days, you
Speaker 2: yourself can win against that system. I can do it
Speaker 2: as well. Why because those systems have stayed the same,
Speaker 2: and humans have identified flaws in that systems. They have
Speaker 2: identified hallucinations errors, And now when you play against that system,
Speaker 2: you just exploit this system, exploit its weaknesses. The system
Speaker 2: is not adapting itself, not rewiring itself. So to answer
Speaker 2: your question quickly, I'm personally not worried of existing AI
Speaker 2: systems because they are adapting themselves. However, I do have
Speaker 2: to mention I do worry that people are going to
Speaker 2: be using existing systems in domains where they should not
Speaker 2: be used, for example, for identifying targets to bombing around.
Speaker 2: It's just don't do those things. Those systems make errors
Speaker 2: and hallucinations. So I worry about misuse of existing AI tools.
Speaker 2: I don't worry about these tools themselves being malicious. No,
Speaker 2: I worry about inadvertent misuse of those tools without understanding
Speaker 2: what they are not good for.
Speaker 1: Okay, interesting, always something to worry about. Can I ask
Speaker 1: you one last thing. One of the things that you
Speaker 1: mentioned earlier was the extraordinary energy inefficiency of the current systems.
Speaker 1: And it is true, isn't it. The human brain is
Speaker 1: remarkably energy efficient. And when you look at these models,
Speaker 1: the amount of energy that they will use to simply
Speaker 1: have the same thought that the human can have on
Speaker 1: a couple of words, it's that's an extraordinary problem that
Speaker 1: in your new generation that we talked about earlier will
Speaker 1: be diminished.
Speaker 2: Absolutely. The systems that I know that fractal brain is
Speaker 2: coming up with, innate AI is coming up with. See
Speaker 2: those systems, they don't use back propagation to produce the
Speaker 2: next word. They don't have to load from the memory
Speaker 2: all of those one point five trillion parameters just to
Speaker 2: produce the next word. They don't do that. They load
Speaker 2: maybe one hundred, maybe one thousand parameters three four orders
Speaker 2: of magnitude better power efficiency. And we know this thing,
Speaker 2: and we know that we've trained what first version of
Speaker 2: our fractal language model using I think zero point one
Speaker 2: percent electricity that open AI use for GPT one and two,
Speaker 2: so we know it's possible to do that. It finds
Speaker 2: its interesting. Before this podcast, we had a conversation. If
Speaker 2: you recall me on Easter dinner and cooking potatoes. So
Speaker 2: as we're talking right now, I can assure you that
Speaker 2: for me and for you to produce the next word
Speaker 2: in our conversation, you are not thinking about potatoes for Easter.
Speaker 2: You're not doing that thing. You don't need those connections too.
Speaker 2: When we talk about a we don't do those things.
Speaker 2: And maybe now you think about your Eastern dinner. But
Speaker 2: the point is that those systems should produce the next
Speaker 2: word only loading up parameters which are important for the
Speaker 2: next world. And this is not billions of parameters, nor
Speaker 2: hundreds of thousands of most maybe a thousand, two thousand parameters.
Speaker 2: So yes, the next generation systems, in addition to being deterministic,
Speaker 2: you can trust their output. They have power and the
Speaker 2: data efficiency which is remarkable, talking three orders of magnitude better,
Speaker 2: it's coming out. This thing is coming better. And again
Speaker 2: it's the world may not be ready for it yet.
Speaker 2: Plus you didn't understand my own motivation for building those
Speaker 2: systems and the dangers. We have systems which are adapting themselves,
Speaker 2: rewiring themselves, so to some extent you cannot outsmart them permanently.
Speaker 2: They'll learn from your mistakes. It's like your kid. Try
Speaker 2: using some technique on your kid. It's going to learn
Speaker 2: how to adapt itself. Those systems do it as well,
Speaker 2: So I personally do not no if it's a good
Speaker 2: time to release those systems to the general public. I don't.
Speaker 2: That's why those systems are not being published because think
Speaker 2: about it. If it has a solution for a system
Speaker 2: which is learning all the time adapting it's flaws, that
Speaker 2: becomes spooky to release that system. So I worry more
Speaker 2: about potential misuse of those next generation systems, less worry
Speaker 2: about their performance because they're already beating existing systems on
Speaker 2: common benchmarks.
Speaker 1: Okay, thank you very much. It's one last thing imish
Speaker 1: before we go. I think our listeners will have been
Speaker 1: absolutely fascinated by all this and what I often ask people,
Speaker 1: I'm going to ask you as well, and I hope
Speaker 1: that we'll be able to understand that.
Speaker 2: Your suggestion, what are you reading at the moment, Well,
Speaker 2: I wish I could tell you the IM reading books
Speaker 2: on AI, but I don't. In fact, four years ago
Speaker 2: I started learning without any necessity Spanish, zero necessity, nothing.
Speaker 2: The reason I started doing that is I wanted to
Speaker 2: again learn a language the way humans do that without
Speaker 2: billions of words, not with a small number of words.
Speaker 2: So as an expers a mental experiment to learn whether
Speaker 2: our fractal language model learns language in the same way
Speaker 2: as I'm learning Spanish, I started learning Spanish, and because
Speaker 2: of that, I'll I'll disappoint you. I'm reading kids' books,
Speaker 2: reading Diary of the wimpik It in Spanish. Of course,
Speaker 2: I'm reading Harry Potter as well in Spanish. I'm reading
Speaker 2: lots and lots of books in Spanish, but these are
Speaker 2: elementary school books. I'm sorry to disappointed.
Speaker 1: You're reading the same things in Spanish that my son is.
Speaker 1: So there we go in common.
Speaker 2: Yes, absolutely, this is actually good because if you want
Speaker 2: to understand, for example, how language works, at least try
Speaker 2: to learn yet another language. Now that you can do
Speaker 2: an introspection, you can see how you're learning a language,
Speaker 2: and so it's remarkable you can learn a language after
Speaker 2: in my case, about seven or eight thousand hours. I
Speaker 2: guess my Spanish is better than my English right now,
Speaker 2: So eight thousand hours, how many tokens we're talking about?
Speaker 2: Couple million tokens, couple million tokens of a training set
Speaker 2: rather than couple trillion tokens. So that's what fascinates me.
Speaker 2: And yeah, maybe next time we can speak in the
Speaker 2: Spanish on the podcast.
Speaker 1: No, I'll have to get one of my kids in
Speaker 1: for that.
Speaker 2: Okay, sounds good.
Speaker 1: Yeahhish, thank you so much for joining us today.
Speaker 2: Its pleasure. Thank you so much for having me.
Speaker 1: Thanks for listening to this week's Marin Talks Money. If
Speaker 1: you like our show, rate to review and subscribe wherever
Speaker 1: you listen to podcasts, and keep sending your questions or
Speaker 1: comments the Merrin Money at Bloomberg dot net. You can
Speaker 1: also follow me and John on Twitter or x I
Speaker 1: met Marinas w and John is Underscore Stepic. This episode
Speaker 1: was hosted by Me Marenzumaset Web was produced by Sammersadi
Speaker 1: and Moses and sound designed by Blake Maples and Aaron
Speaker 1: Kasper and special thanks of course to Yannish Mareski.