S1: Welcome to Unsupervised Learning, a security, AI and meaning focused
S1: podcast that looks at how best to thrive as humans
S1: in a post AI world. It combines original ideas, analysis,
S1: and mental models to bring not just the news, but
S1: why it matters and how to respond. Hey what's up?
S1: It's Daniel with Unsupervised learning. I'm building AI to upgrade humans,
S1: and today I want to talk about a cool idea.
S1: Basically trying to figure out how to predict how much
S1: AI infrastructure we actually need and how far along we
S1: are along that path. And I want to say something
S1: about predictions real quick, which I've talked about before in
S1: other videos, but I don't think it's possible to predict
S1: tech really, the way that tech happens, who wins, who loses,
S1: like the timelines? Like this stuff is like impossible to predict, basically.
S1: So I wouldn't want you to think that I would
S1: try to do that because I think that's foolish. What
S1: I believe we can predict related to tech are the
S1: human things that are related to human desires. So the
S1: question is not whether we can predict tech itself, but
S1: can we predict what we want from tech. And I
S1: think that is a powerful way to kind of see
S1: what might be happening and what could unfold in the
S1: next 1 to 3 years or whatever. So I want
S1: to specifically look at like, how far along are we
S1: to building the infrastructure that we need for AI, right. So, um,
S1: I own a bunch of, uh, Nvidia, um, I think
S1: I've got some TSMC like, um, I've got stock related
S1: to AI. I'm all in on AI, have been since
S1: late 2022. Uh, I went independent to work in this field, um,
S1: after being in security for like 25 years. So I'm very,
S1: I guess, religious about the whole thing. So I want
S1: you to understand that which you probably already know if
S1: you've seen any of my videos? Um, but I don't
S1: think that affects this particular thing that we're talking about.
S1: I'm sure it does in some sense, because that's the
S1: way bias works. But I think this argument I'm about
S1: to make stands independent of that. So essentially what I'm
S1: looking to figure out is how much infrastructure do we need?
S1: How many GPUs do we need? How many data centers
S1: do we need? How many AI companies and startups do
S1: we need? And some of that depends on the actual
S1: technical implementations, which we can't predict. So we don't need
S1: to go too deep into that. But I'm just trying
S1: to figure out, are we at like 13% of how
S1: much I. We need a lot of people think we're
S1: at 85%. For example, they're like, yeah, that was pretty
S1: much it in 2023 and 2024. And now it's mostly
S1: hype and it's just going to die down. So they
S1: think it's like 87% and it's pretty much done. It
S1: was a cycle. Whatever other people are like no we're
S1: just getting started. We're at like 3%, but it's going
S1: to grow over the next five years or whatever. And
S1: I guess I'll just spoil what I think the answer is.
S1: I think the answer is we're at like 0.000. I
S1: don't know how many zeros, like eight zeros and a 1%
S1: of where we need to be or where's more specifically
S1: where humans will demand that we get. Okay. So that's
S1: that's the point I want to make here. And the
S1: way I want to get there is by asking a question,
S1: what are we actually trying to do with AI? What
S1: are we trying to do with AI? So I want
S1: to walk through a few ideas there. So one cool
S1: idea here is learning from the past and anticipating the future.
S1: So we are sitting in the middle here in the
S1: current present moment, and we could use AI to look
S1: into the past, find things that went wrong for the
S1: purpose of adjusting our current behavior right on the other side.
S1: We could anticipate, try to anticipate Paint based on everything
S1: we currently know. Try to anticipate what's going to happen
S1: in the future. For what reason? Same exact reason. Adjust
S1: our current behavior. So I think this is a paradigm.
S1: This is an idea or a concept that's extremely powerful.
S1: And so the question is how much more of this
S1: do we need? How much more of this do we need?
S1: Or how bad of a job are we currently doing
S1: at this? And how much more can we gain if
S1: we were doing it much, much better? So I won't
S1: even give my answer, but probably kind of rhetorical. All right,
S1: next idea I would say that for any given situation,
S1: for any given operation or thing you're trying to do,
S1: there is a current state and there is a desired state,
S1: and there is a delta between those two. And I
S1: think this is a really powerful concept. It automatically asks
S1: a question. It demands a question how do we get
S1: from the current state to the desired state? So I
S1: ask you again, how many situations do we have worldwide,
S1: human civilization wide, where we have a current situation that
S1: we're not really happy with, and we either need help
S1: coming up with a desired state, or we already have
S1: a desired state clearly in mind. Or maybe we have
S1: thoughts about a desired state, but we could use some
S1: help articulating it. But either way, we want to get
S1: to this desired state, which I could potentially help us
S1: articulate the desired state. But either way, how are we
S1: doing on understanding our current state perfectly, articulating our desired state,
S1: and figuring out the delta between those two? Think education. Poverty. Climate. Health. Aging. Politics. War.
S1: Just just think all the whole scope of human problems
S1: over the course of history. How are we doing on this?
S1: How much potential is there to do this better? That's
S1: the second one. And the second one really brings up
S1: this third one. What are the actions that we should
S1: take to make the transition? This this is incredibly powerful.
S1: Can we use AI to understand the current state? Can
S1: we use AI to articulate and structure and build and
S1: communicate the desired state? And most importantly, what's the plan?
S1: How do we transition from one to the other? How
S1: much do we need this middle piece? How good of
S1: a job are we as humans doing at executing on
S1: this middle piece? I would argue not very well at all.
S1: And we haven't been for centuries. Well, I mean, you
S1: could argue it could go way worse. Totally. Absolutely could
S1: go way worse. So sure, we're not all dead yet,
S1: so that's good. But if you look at the course
S1: of history, it's just like folly after folly. And we're
S1: currently in a whole bunch of it right now. And
S1: it's a bad situation to be in. So I would
S1: argue that this yellow piece right here in the middle
S1: is extraordinary. It is. It has extraordinary potential for what
S1: we wish we could do with that yellow to transition
S1: to this desired state. So let's break these three down
S1: even more. When you look at the current state, you
S1: talk about, okay, let's understand the current state. How granular
S1: are we talking about? Right. Because you could go from
S1: like a coffee cup with like an amount of hot
S1: liquid inside of it. And we could ask the question,
S1: what is the current state of this liquid inside this
S1: coffee cup? What's the current state of the coffee? Well,
S1: you could ask lots of different people about this. You
S1: could ask a philosopher, you could ask whatever. It's like, oh,
S1: half full, half empty. Um, you could ask like a
S1: chemist or a physicist. And it's like, well, um, what
S1: do you want to know? Like how many atoms, like,
S1: how excited are they? What the the location of all
S1: the electrons. Certain parts of this question for a given
S1: object are just not knowable. Like, what is the state
S1: of all atoms and subatomic particles in, say, a pen
S1: or in a cup of coffee or the state of
S1: a human, for example, you say, how is he doing?
S1: How is she doing? Well, what do you mean? I
S1: can't give you a full rundown of every atom in
S1: their body. So the question is, what level of depth
S1: do we need to be able to answer that question?
S1: And that comes down to how much telemetry can we get,
S1: how much data can we get coming off of this situation.
S1: So that's I've got an aura ring on, I've got
S1: an Apple Watch on. This is telemetry coming off of
S1: me so that you can ask an AI very soon
S1: or actually now in my opinion, Apple is building life
S1: OS essentially, in case you guys didn't know that. So
S1: you will be able to ask yourself, how am I doing? Um,
S1: and it will know. Okay, you're talking about health. You're
S1: talking about financially, you're talking about education. What are you
S1: talking about? But you have to have that telemetry coming in, right?
S1: And you have to know what the limits are right for.
S1: For a given object type, you have to know certain
S1: questions and certain types of metrics that are just attainable. Okay.
S1: Can we get someone's current heart rate? No, not 15
S1: years ago. Not automatically, not every moment of the day.
S1: We couldn't do that before. We can now. So that's
S1: a technology change that enabled that level of metric to happen. Well,
S1: Apple is currently gathering things like mood as well. How
S1: are you feeling today. And it's associating those things with
S1: other things like how much you've exercised and stuff like that. Right.
S1: So you kind of see where this is all going
S1: with life. OS now that's for a person. Okay. Now
S1: let's talk about a family. What's the state of the family? Well,
S1: what does that mean? Do we need all of those
S1: metrics for each individual person? Sure. But you also want
S1: to know the dynamics between the people. You also want
S1: to know how the family as a whole is doing. Um,
S1: what's the quality of life? What is the upward trajectory
S1: look like? Right? This is the state of a family.
S1: Now let's talk about business companies. What is the state
S1: of a company? Well, what are we talking about? Its
S1: IT infrastructure with thousands of Kubernetes pods running. What's the
S1: current state of a Kubernetes pod right now? And also
S1: right now and also right now, how often are we
S1: polling these things both for a human and for a company? Right.
S1: And a lot of these questions are going to come
S1: down to, sure, we could poll the current state of
S1: every single Kubernetes pod at, say, I don't know, Google, right?
S1: Every single second, but that might cost billions of dollars
S1: because it takes, you know, it's just expensive. And where
S1: would you store the stuff in all these different questions.
S1: So you've got like how granular, what is the update
S1: frequency and how expensive is that to gather based on
S1: the current tech. Right. And this is Tremendously interesting and
S1: tremendously powerful, especially for something like a human. The most
S1: important driver for all of this AI stuff one is
S1: going to be business, no question. Right? And that's going
S1: to have the money behind it or whatever. But ultimately,
S1: the bigger thing is going to be humans using this
S1: to feel better and be better and improve themselves. That's
S1: that's the entire game here. What are people struggling with
S1: right now? Job happiness. Can't get a job. Loneliness. A
S1: lack of meaning in their lives. Okay. How do you
S1: solve how how do you solve the current state desired
S1: state situation for somebody who's lonely? And I got another
S1: example of that coming up. But think about that. Think
S1: about what it takes to sort of solve these problems.
S1: Go from current state, desired state for a human, for
S1: a company, and what level of context you need to
S1: be able to answer those sorts of questions. now, I
S1: would argue just like the previous ones, we are nowhere
S1: near gathering enough context on the current state of anything.
S1: Of anything. Okay, a park bench, a tree and at
S1: the park, the state of a human that you care about.
S1: We don't have nearly enough telemetry on that. We want
S1: to change a business. We have no idea what's going
S1: on in business. Most people who work there and actually
S1: run the business, they have no idea what's going on
S1: in the business. They don't know the level of mood
S1: and happiness and the state. How much waste is happening
S1: at any given time? Bottom line here is things like
S1: context size, things like Rag technologies like these are absolutely
S1: in their infancy because they have to be because of
S1: the size of the scope of the problem that they
S1: actually are being signed up to solve. Okay. Uh, Altman
S1: talked about this a while back. He was like, look,
S1: at some point, context is basically going to be infinite,
S1: and obviously nothing's infinite. So that doesn't really mean infinite.
S1: It means functionally infinite. So there's going to be this
S1: competition between context size going into models versus Rag, going
S1: into models versus whatever, whatever comes out later and makes
S1: maybe those not as important. But either way, what you
S1: have to do is you have to get this current
S1: state for a thing that matters to humans. Forget tech
S1: is about a thing that matters to humans into the
S1: brain of the AI, so that all it's understanding of
S1: the world can be applied to this larger situation. Now
S1: think about this as a human. Imagine somebody who's 40
S1: years old and has had this amazing life and, you know,
S1: the hardships and people died and they fought in wars
S1: and they traveled to Costa Rica and they did ayahuasca,
S1: and they learned all these things. They know thousands of
S1: people and they've had this wonderful life, and they're trying
S1: to craft even a better life. And you want to
S1: ask an AI, you know, tell me what I can
S1: work on. Tell me about myself. What do you suggest
S1: for me? I'm looking at doing this career or this career.
S1: I'm looking at moving here. I'm looking at marrying this
S1: girl or not marrying the squirrel. Um. What should I do?
S1: What does the. I need to know about that person
S1: to make that decision? Or to offer advice in making
S1: that decision? Ideally, it has everything. It's got the genome.
S1: It's it's got all your medical records growing up. Ideally
S1: it has journal entries. Ideally, it's been recording you at
S1: a deep level of state gathering for your entire life.
S1: So imagine this is 200 years from now and this
S1: person is 40 years old. But every moment of their
S1: life has been captured. And the very moment that you
S1: ask a question, what should I do? Which career? Which
S1: country do I move to? Which guy do I marry? Um,
S1: the entirety of their life up to that moment would
S1: be the context for answering that question. This is the
S1: key point for this entire thing. Think of the AI
S1: infrastructure that is required to have the entire context of
S1: the moment, right up to the point when the question
S1: is asked to be jammed into that AI's mind. But
S1: don't just imagine it for this 40 year old guy
S1: in Costa Rica. Imagine it for a giant business with
S1: 10,000 employees who's been in business for 120 years, how
S1: much context, at a deep level, can we grab from
S1: that entire 120 years and put it into what should
S1: I do now that is big. That is massive. And
S1: here's the crazy part. It can be summarized to a
S1: ten page text document. It can be summarized to a
S1: 1010 page text document. And guess what? It would be
S1: good if you could summarize it to a 200 page report.
S1: It would be really good. It could also be terabytes
S1: of data. Petabytes of data. How quickly can you get
S1: petabytes of data into an AI's brain to snap and
S1: answer the question in a really, really powerful way for
S1: a business, for a country, for a city, for a human,
S1: for a family. Think of the infrastructure that is required
S1: to do that, and think about the fact that this
S1: is what humans want. This is what humans will demand.
S1: This is why I'm saying forget the tech. Forget the
S1: tech does not matter. What matters is what humans want.
S1: Humans want the most amazing answer ever to that question.
S1: Do I do this merger with this company? Do I
S1: hire this person as my CFO? And the amount of
S1: data that is needed to get a better and better
S1: and better answer to that question just keeps going up
S1: along with the capabilities of the tech. Right. So this
S1: is what's so powerful about this paradigm of thinking about
S1: this is that there's kind of no end. There's no
S1: foreseeable end coming to how much better it gets when
S1: you give it more. And that's why okay, 200 K
S1: tokens right now. So that's a lot. No it's not
S1: it's not a lot. It's not a lot because watch this.
S1: The thing a human wants to do is to say hey,
S1: I've got an idea for a concept for this really cool, um,
S1: mini series that's going to be on Netflix. It's got anime,
S1: it's got Harry Potter stuff, but it's in this stylized thing,
S1: kind of like The Last Samurai or, um, or what
S1: was that? It was called Shogun, the recent animated thing. Anyway,
S1: you give it all these things, you're like, I want
S1: this art. I want the concept of Star Wars, but
S1: it's got to be Harry Potter. But I also want
S1: like deep romance. But I want it to be really,
S1: really gritty. And it definitely, you know, got to be
S1: over 21 to even watch this thing. And there's also
S1: violence and there's also sex and there's also all these
S1: different things. And you basically hit it with that and
S1: you're like, yeah. But I also want it to be
S1: kind of like Tolstoy. I really like Tolstoy. And it's
S1: going to go and write you and build you a
S1: complete movie, a complete series, and do all these different
S1: things with agents, all the stuff that's coming out. That's
S1: kind of inevitable, right? How much does it need to
S1: know to be able to do that perfectly? It's got
S1: to be able to read into everything you're saying. It's
S1: got to have all the capabilities of actually doing the
S1: video and the art creation and all this stuff, but
S1: the more it knows, the better this thing gets. And
S1: that doesn't really stop at any time soon. Right? So
S1: this is needed for business. This is needed for creativity.
S1: This is needed for human thriving at an individual and
S1: a society level. There is no end to how much
S1: we will demand this thing. I'll give you another example here.
S1: Because of this assassination that happened with the head of United,
S1: UnitedHealth and I talked about this in in my book,
S1: The Real Internet of Things in like 20, 2016. It
S1: came out and I've recently done a whole series on
S1: like modernizing it. And it's got video and everything, so
S1: you should check it out. But the point is, what
S1: I talked about in that book was walking down the
S1: street and having sensors all around you. This is the
S1: vision that I'm seeing for everything. You have sensors all
S1: around you, the current state of the world. This thing
S1: on the left here, this current state of the world,
S1: is the thing that your eye is always monitoring for you.
S1: Your Da, your digital assistant. Right? It could see behind you.
S1: You could see above you. Why is it seeing above you?
S1: This is dystopian, but this is real. Because of drones,
S1: that's why. Why is it seeing behind you? Because someone
S1: might walk up behind you with a big backpack on
S1: and a mask and pull out a gun and shoot you.
S1: So what your eye has to be doing is watching
S1: everything all the time. And guess what? If you're a
S1: VIP or you just have some money, you're going to
S1: have little drones flying around looking at everything, looking down. Hey,
S1: why is that car moving like that? Hey, moved. Move
S1: to the left across the street. I don't want you
S1: over here. Your little ear piece is going to be
S1: giving you these little guidance things. It hears your stomach rumble.
S1: It's like, hey, there's Thai food up there. It's your
S1: favorite restaurant. I already pinged Jeremiah. He's got your favorite table.
S1: I turned on table tennis. So when you get there,
S1: table tennis is on. It is monitoring the state of
S1: the world and changing the state of that world in
S1: preemptively watching for the state of the world to make
S1: sure it doesn't turn into an undesired state for you
S1: 24 over seven like every second. Boom, boom, boom boom.
S1: It's reading every API. It's looking at every person. It's
S1: pulling up their daemon to see if there if there's
S1: information on them, like should we go introduce ourselves to them?
S1: This is the parsing of the current state of the
S1: world around us at all times. That once again only
S1: gets better and better with more tech, right? You could
S1: do this a little bit in 2025 and 2026 and 27.
S1: That'll be pretty good. But wait till you see it
S1: in five years. Wait till you see it. In ten
S1: years it will be absolute sci fi stuff. And again,
S1: you don't need to think about the tech. You only
S1: need to think about the fact that humans wish they
S1: could see around themselves at all times. They wish they
S1: had that tech on all of their kids at all times.
S1: So when some white kid jumps out of a jumps
S1: out of a pickup in the front of the school
S1: and runs into the thing, and he looks like he's
S1: over 18, but probably 19 or 20, and he's got
S1: a backpack. Suddenly the earpiece in your kid's ear and
S1: in the ear of the principal and everyone else. Boom!
S1: There's data happening. Why? Because there's drones flying over. This
S1: is a this is a monitored state of the world
S1: that I parse and hand to the people who care
S1: about that state. And going back to the first point,
S1: there is so much you can gather every everything's going
S1: to have an API. You're going to be able to
S1: pull all this data. But okay, why am I pulling
S1: gigabytes of data all the time? Can I even process it?
S1: The answer is no, you can't, which means we won't
S1: pull that yet. But that will push us to be
S1: able to process it. Oh, now we can. Now we'll
S1: pull more data. This will just keep ratcheting up and
S1: keep ratcheting up. And until until you eventually have something
S1: like Neuralink and you just kind of know the state
S1: the same way you see color. You will know the
S1: state of danger. Right? And I believe all this is inevitable,
S1: but it's so far in the future. Like you don't
S1: need to worry about it. But what I'm trying to
S1: do is show you that it's obvious that that sort
S1: of push is happening from humans, because that's what we demand, right?
S1: So that's kind of where we're going, um, and how
S1: it will happen and who will do it or whatever.
S1: Who knows? Nobody knows. Could be a giant single corporation
S1: that does everything. And or it could be a million
S1: different startups. Who knows? It's impossible to predict. The point is,
S1: we will demand this and we will demand it gets
S1: better and better. And the deeper you go on how
S1: granular and the more often the updates to the frequency
S1: are demanded, and the more the cost goes down, the
S1: more we will have. It's this thing just goes crazy. Okay,
S1: now we talk about what's the desired state. Now we
S1: talk about how do we get from here to there?
S1: What's the planning? Right? How do we make a I'm
S1: reading good to great right now. How do we make
S1: this business from good to great. What are the changes
S1: we make. Right. That's also an AI eye piece. All
S1: of these, all of these combined are going to push
S1: this thing from us barely starting this year, last year,
S1: year before to it's not a hockey stick. It is
S1: straight up into the sky in how much context we
S1: need and how much memory we need and how much
S1: processing we need right now. It could be that new
S1: processors come out so we could do a 100,000 times
S1: as much processing, with 100,000 times less effort and resources
S1: and stuff like that. Doesn't matter. We'll just ask for
S1: 100,000 times more. We'll be like, oh, cool. So so
S1: you can pull, um, 40,000 metrics on me as a
S1: human every second. Cool. Can you get the state of
S1: every molecule in my body? And they'll be like, no,
S1: that's impossible. Cool. Someone will go work on it. And
S1: suddenly the current hardware is not good enough. And we
S1: could make better predictions if we knew the state of
S1: every molecule in the body. And guess what? It just
S1: keeps going. It just keeps going. That's the point of this.
S1: So these are the three basic pieces current state, desired state,
S1: and the actions, processes and SOPs. SOP is a standard
S1: operating procedure. I think this is a really, really powerful concept.
S1: I think these three combined are going to be like
S1: a framework for making improvements to things. I did a
S1: piece in early 2023, like March of 2023, called Pspca
S1: State Policy Questions and Actions. State is state of the world.
S1: Exactly like this policy is the desired state of your
S1: company or your life, or whatever questions is like what
S1: are we asking to this I body of knowledge and
S1: then action is what do we do as a result
S1: of learning the answer. Right. So I think that's a
S1: really powerful structure for thinking about how to improve anything,
S1: and it's all based on the stuff we've been talking about.
S1: So some examples of this. A lonely human wants to
S1: be a happy human. So the recommendation is going to
S1: be to connect. And all of these are going to
S1: be extremely granular with lots of different recommendations going all
S1: the way down. Right. And of course this will all
S1: be managed by a Da AI of some sort. You've
S1: got a dying business. You wish you had a thriving
S1: business that was actually making money. What are you going
S1: to do? You're going to increase revenue by doing the
S1: following things. This flow here of understand the current thing,
S1: understand where you want to be and have AI figure
S1: out the stuff in the middle. At first it's going
S1: to be recommending things to us right as it is now,
S1: but very soon and already starting. It's going to just
S1: be like, do you want me to do that for you?
S1: And the businesses are going to be like, yes, absolutely.
S1: That means I don't have to hire anyone. In fact,
S1: that means I can get rid of a lot of
S1: people because you could just do that for me. So
S1: what are the predictions that we can make based on this?
S1: I think of the three. The state piece is the
S1: most important. Just because you can't really do anything unless
S1: you understand what's currently happening, right? It's hard to it's
S1: hard to talk about improvement if you don't know what's wrong. Um,
S1: so state of the human state of a company. State
S1: of a family. Cup of coffee. We've talked about this
S1: infinitely complex changing constantly, and that's why the stuff is
S1: going to scale so crazily, in my opinion. And who
S1: knows if Nvidia is going to win. I think they'll
S1: probably win for at least a couple of years, but
S1: I have no idea they could crash tomorrow. Or they
S1: could whatever, go to the stratosphere in six months? No idea.
S1: So this this brings us to the question how much
S1: context size do we need? How many tokens do we
S1: need to be able to put into context? How big
S1: are Rag systems going to get? How many GPUs do
S1: we need? How many startups do we need in the
S1: AI space? How much I do we actually need? That
S1: is ultimately what we're trying to answer here. And my
S1: answer to that is enough to constantly poll the state
S1: of everything we care about, and then take actions to
S1: fix it to get it into the desired state. And
S1: I think that is a metric crap ton of AI.
S1: And that's why I think we're at like, like I said,
S1: point ten zeros and a one on where it's actually going.
S1: See you in the next one. Unsupervised learning is produced
S1: and edited by Daniel Miessler on a Neumann U87 AI
S1: microphone using Hindenburg. Intro and outro music is by Zomby
S1: with a Y, and to get the text and links
S1: from this episode, sign up for the newsletter version of
S1: the show at Daniel missler.com/newsletter. We'll see you next time.