Speaker 1: Ze Media.
Speaker 2: Hi, my name said Tron, and welcome to Better Offline.
Speaker 2: This is also Jackass. So you've just had a cheery
Speaker 2: two part chuckle first about how Generative Ai made tanker
Speaker 2: markets in our economy. So I'm going to give you
Speaker 2: a lighter one an episode about GPT five, which is
Speaker 2: a model from open Ai, and why just under three
Speaker 2: years of hype have led to the software equivalent of
Speaker 2: the launch of Saint Anger, except every time lars are hit.
Speaker 2: The snare drama cost them fifty five thousand dollars. Now,
Speaker 2: if we look at the positive reviews, we see takes
Speaker 2: ranging from Simon Willison's tempered remark that GPT five is
Speaker 2: just good at stuff to semi anass this is completely
Speaker 2: insane statement that GPT five is setting the stage for
Speaker 2: ad monetization and the open Ai GPT chat GPT super app.
Speaker 2: In a piece that makes several assertions about how the
Speaker 2: router that underpins GPT five is somehow the secret way
Speaker 2: that Openaye will inject Dad's which is just distinctly silly.
Speaker 2: It's I'll get into this in the episode a little bit,
Speaker 2: but just with everything you're going to hear, you're going
Speaker 2: to realize that this is just someone just saying stuff.
Speaker 2: Took four bylines to do that shit too. I'm also British.
Speaker 2: I'm gonna say router. I might say router as well,
Speaker 2: because I've been here a while. Make fun of my
Speaker 2: voice if you really must. But with that out the way,
Speaker 2: here's a quote from semi Analysis' coverage. Before the router,
Speaker 2: there was no way for a query to be distinguished,
Speaker 2: and after the router, the first low value query could
Speaker 2: be routed to a GBT five mini model that can
Speaker 2: answer with zero tool calls and no reasoning. This likely
Speaker 2: means serving this user is approaching the cost of a
Speaker 2: search query. This does not make any sense. This None
Speaker 2: of this makes it like it's just a bunch of assumptions.
Speaker 2: Why would this be the case. The article also makes
Speaker 2: a lot of claims about the value of a question
Speaker 2: and how chat GPT could I am serious a agent
Speaker 2: agentically reach out to lawyers. I'm not going to edit
Speaker 2: that out because egentically is not a fun word to say.
Speaker 2: It is just complete nonsense, and in fact, I'm not
Speaker 2: sure this piece reflects how GPT five even works at all. Again,
Speaker 2: quoting it, the router serves multiple purposes on both the
Speaker 2: cost and performance side. On the cost side, routing users
Speaker 2: to many versions of each bubble allows open ai to
Speaker 2: service uses at a lower cost or with lower costs.
Speaker 2: Even to be fair on semi analysis, it's not as
Speaker 2: if open ai gave them much help. Open AI's official
Speaker 2: writings about the router aren't exactly filled with details, talking
Speaker 2: and glowing terms about what it does, but not how
Speaker 2: here's what they say. Chat GPT's real time router quickly
Speaker 2: decides which model to use based on the conversation type, complexity,
Speaker 2: tool needs, and your explicit intent. For example, if you
Speaker 2: say think hard about this in the prompt. The router
Speaker 2: is continuously trained on real signals, including when users switch models,
Speaker 2: preference rates for responses, and measured corrected correctness improving over time.
Speaker 2: Once usage limits are reached, a mini version of each
Speaker 2: model handles remains inquiries. In the near future, we plan
Speaker 2: to integrate these capabilities into a single model. And that
Speaker 2: last bit really doesn't make sense, but in any case,
Speaker 2: the lordchip GPT five has been very, very weird. At first.
Speaker 2: Some people seemed really happy about it. Chief of them
Speaker 2: software YouTube of Theo Brown, who is over four hundred
Speaker 2: and sixty eight thousand subscribers. He's also known as theogg
Speaker 2: who said.
Speaker 1: I didn't know it could get this good. This was
Speaker 1: kind of the like oh fuck moment for me in
Speaker 1: a lot of ways, and I've had to fight like
Speaker 1: a slow spiral into insanity. It's a really really good model.
Speaker 2: He finished by saying, and.
Speaker 1: Keep an eye on your job because I don't know
Speaker 1: what this means for us long term.
Speaker 2: Pretty crazy, right. Comments on the video included people saying
Speaker 2: things like if open aye is helding you hostage, blink
Speaker 2: twice and yes that is an adverbating quote. Another saying
Speaker 2: this dude, is everything wrong in it today? Another saying
Speaker 2: this video was sponsored by open Ai, Another other saying
Speaker 2: ge GPT five failed every test project I gave it today.
Speaker 2: It's a lie in my experience. Maybe they haven't ramped
Speaker 2: up the GPUs now. From what I can tell, THEO
Speaker 2: Brown played with GPT five in open ais offices and
Speaker 2: did all the benchmarking there. Open Ai, by the way,
Speaker 2: fucking how come on? You can't benchmark in their offices anyway.
Speaker 2: Open AI's API based access to GPT five models. You
Speaker 2: know the thing that you use if you want to
Speaker 2: integrate GPT into your app, does not root them, by
Speaker 2: the way, nor does open ai offer access to its
Speaker 2: router or any associated models. Important detail. Just want you
Speaker 2: to know that because we need to make sure very
Speaker 2: clear now A weekly a Theo Brown would put out
Speaker 2: another video called I was wrong about GPT five, which
Speaker 2: he would open by saying.
Speaker 1: So first and foremost, I want to make sure it
Speaker 1: is very very clear that the experience that you probably
Speaker 1: are having with chat, GPT and GPT five right now
Speaker 1: is not the experience that I had when I was
Speaker 1: first testing it.
Speaker 2: Brown goes on to explain that he was not paid
Speaker 2: by open Ai at all, that he was sincerely impressed
Speaker 2: by the company and GA five, and that he'd actually
Speaker 2: spent over twenty five thousand dollars in inference testing it
Speaker 2: on his own company software, and indeed also that he
Speaker 2: turned down a grand appearance fee. Sorry, I mean that's
Speaker 2: a very British thing, one thousand dollars appearance fee, not
Speaker 2: just like a really nice one. Brown claims he asked
Speaker 2: open Ai to try it out, and after they declined
Speaker 2: to let him test it early on his own, he
Speaker 2: was invited to try it on camera with a small
Speaker 2: group of other people open AI's offices where they'd film
Speaker 2: his reactions. He said that the API was incredible, but
Speaker 2: that it's become apparent that the models he was using
Speaker 2: in the video were not the same as those released
Speaker 2: of the public. Making a post on August thirteenth on
Speaker 2: xd Everything app that GPT five was nowhere near as
Speaker 2: good as in cursor as when it was as it
Speaker 2: was when he was using it a few weeks ago,
Speaker 2: complaining that things that worked while demoing it at open
Speaker 2: ai no longer did, adding that there was something somebody
Speaker 2: else on Twitter that said they'd had a similarly great
Speaker 2: experience GPT five on launch that has since decayed. It
Speaker 2: isn't completely clear what happened here, but I'm going to
Speaker 2: guess that open ai showed THEO Brown and others in
Speaker 2: their offices some sort of heavily molded version of the
Speaker 2: model that burns significantly more compute to provide its outputs,
Speaker 2: though I'm also very suspicious of how significance the difference
Speaker 2: is here. Brown's videos attempt to show the difference between
Speaker 2: the generations that you received from the model when it
Speaker 2: was good and when it was bad. In this video,
Speaker 2: which I'll include a link to in the episode notes.
Speaker 2: But if I'm honest, they look pretty similar in that
Speaker 2: they're kind of mediocre. I'm not saying that as a hater,
Speaker 2: by the way. They just kind of look like shit.
Speaker 2: It's just kind of okay, like shit. They look like
Speaker 2: regular fucking generated websites. They don't look special. The good
Speaker 2: one is fine, and the bad one has weird gradients
Speaker 2: on it. This whole thing sucks, though, and was a
Speaker 2: clear set up by open Ai to overstate the abilities
Speaker 2: of GPT five, one that fell apart with the lightest
Speaker 2: brush with reality. I imagine their assumption was that Brown
Speaker 2: would post the glossy video and then walk away, and
Speaker 2: it gave THEO some credit for straight up stating he
Speaker 2: was misled. This was a desperate move and one that
Speaker 2: blew up in the face of open Ai. Along with
Speaker 2: the rest of the GPT five launch. People hate the model,
Speaker 2: customers are mad for taking models away like four to
Speaker 2: H and have remained mad even with their return, and
Speaker 2: the chat gpt subreddit is almost entirely people complaining about
Speaker 2: how ineffective the new version is and how even GPT
Speaker 2: four ROH is not the same They got game of
Speaker 2: brain Baby. As I said in last week's monologue. I
Speaker 2: believe open Ai has grown a fandom rather than any
Speaker 2: kind of sustainable product market fit, and they're now suffering
Speaker 2: fandom like hate with every minor change they make in
Speaker 2: an attempt to push GPT five further, further aggravating people
Speaker 2: that barely understand why they use the product to begin with. Yeah,
Speaker 2: the center of the angle laid the reason for GPT
Speaker 2: five's launch, the belief that this was somehow a cost
Speaker 2: cutting measure, where OpenAI had added a router to chat
Speaker 2: GPT as a means of sending certain requests to cheaper
Speaker 2: models to save money. But when I hear router, I
Speaker 2: hear latency, and I never or even a second believe
Speaker 2: that this would somehow be cheaper to run. It didn't
Speaker 2: make sense. I'm a curious little criator, so I went
Speaker 2: and found out how chat GPT five actually works, and
Speaker 2: unlike the following incredible products that you should buy, it's
Speaker 2: actually kind of a big piece of shit. And we're back,
Speaker 2: and from here on out, I will define two things.
Speaker 2: GPT five referring to the model and its associated mini
Speaker 2: and nano models, and Chat GPT five referring to the
Speaker 2: current state of chat GPT, which features an auto fast
Speaker 2: and thinking and thinking mini model selections. You also can
Speaker 2: see legacy models, but that's not what we're talking about today,
Speaker 2: and that's also only for a little bit. It's a
Speaker 2: distinction I have to make, by the way, and make earlier,
Speaker 2: because the two things are different, they work in different ways,
Speaker 2: and chat GPT five structure induces a bunch of trade
Speaker 2: offs and downsides that, as I'll discuss later, make this
Speaker 2: whole thing even more wasteful. In discussions with a source
Speaker 2: that an infrastructure provider familiar with the architecture, it appears
Speaker 2: that chat GPT five is in fact potentially more expensive
Speaker 2: to run than previous models, and due to the complex
Speaker 2: and chaotic nature of said architecture, can at times spun
Speaker 2: upwards of double The tokens per quid tokens, for those
Speaker 2: who don't know, are basically chunks of texts that the
Speaker 2: AI models do stuff with. I'm simplifying this. Do not
Speaker 2: email me and correct some minor thing nobody cares. A
Speaker 2: sentence like the quick brown fox jumps over the lazy
Speaker 2: dog will be broken into lots of smaller four character chunks.
Speaker 2: There are different kinds of tokens, and they're all priced differently.
Speaker 2: An input token refers to the data you send to
Speaker 2: the model when you ask a question. Output tokens are
Speaker 2: used to measure the size of its response, with bigger
Speaker 2: responses requiring more tokens. The more tokens you burn paquery,
Speaker 2: the more expensive it is to run that query. The
Speaker 2: fact that chat GPT five can, in certain circumstances burn
Speaker 2: twice the number of tokens of query means that every
Speaker 2: question costs more. Chat GPT is also significantly more convoluted,
Speaker 2: plagued by latency issues, and is more compute intensive thanks
Speaker 2: to open a ey's new, smarter, more efficient model routing system.
Speaker 2: In simpler terms, every user prompt on chat GPT, whether
Speaker 2: it's in auto, fast thinking or Thinking Mini, starts by
Speaker 2: putting the users prompt before the static prompt. I don't
Speaker 2: want to lose you here. This is important. A static
Speaker 2: prompt is the invisible instructions given by open Ai to
Speaker 2: chat GPT, in the models themselves and the tools associate
Speaker 2: with them to tell them how to operate. Instructions like
Speaker 2: you are chat GPT, you're a large language model, You're
Speaker 2: a helpful chat bot. Do not threaten them with a knife,
Speaker 2: and so on and so forth. These static prompts are
Speaker 2: different with each model you use. A reasoning model will
Speaker 2: have a different instructions set to a more chat focused one,
Speaker 2: such as think harder about a particular problem before giving
Speaker 2: an answer. Break down problems into component answers. When you
Speaker 2: get a certain thing, like if someone asks you a
Speaker 2: coding question, query a coding tool. That kind of thing,
Speaker 2: a user prompt is exactly what it sounds like, the
Speaker 2: thing that a user wants the AI model to do.
Speaker 2: The new order in chat GPT five becomes an issue
Speaker 2: when you use multiple different models in the same conversation.
Speaker 2: Because the router, the thing that selects the right model
Speaker 2: for the request, has to look at the user prompt.
Speaker 2: It can't consider static instructions first because they may be
Speaker 2: different based on what the user asked. In fact, the
Speaker 2: order has to be flipped for the whole thing to work.
Speaker 2: But simpler previous versions of chat GPT would take the
Speaker 2: static prompt and then invisibly append the user prompt onto it.
Speaker 2: This static prompt would typically be cashed massively, reducing the
Speaker 2: amount of compute the model needs to perform a task.
Speaker 2: Chat GPT cannot do this. Every time you use chat
Speaker 2: GPT five. Every single thing you say or do can
Speaker 2: cause it to do something different. Attach a vile might
Speaker 2: need a different model. Ask it to look into something
Speaker 2: and be detailed. Might trigger a reasoning model or a
Speaker 2: different depth of reasoning. Ask a question in a weird way. Sorry,
Speaker 2: the route is going to need to send you to
Speaker 2: a different model entirely each time, coming up with new
Speaker 2: instructions based on the subtle interpretation of what you asked in.
Speaker 2: Every single thing that can happen when you ask chat
Speaker 2: GPT to do something may triget the route to change model.
Speaker 2: A request a new tool, and each time it does
Speaker 2: so requires a completely fresh static prompt, regardless of whether
Speaker 2: you select auto thinking Faster or any other option on
Speaker 2: chat GPT. This in turn requires it to expend more
Speaker 2: compute with queries consuming more tokens compared to previous versions.
Speaker 2: It's like you started a job, and every time you
Speaker 2: do a task, right an email, make a cup of copy,
Speaker 2: attend a meeting, email someone with a threat your workplace
Speaker 2: requires you to complete the entire mandatory onboarding training first.
Speaker 2: One way that it is spreadsheet, not before you brush up
Speaker 2: on your anti biberary legislation. First your prick. As a result,
Speaker 2: Chat GPT may be smart, but it doesn't really seem
Speaker 2: efficient in the GPT five version. Now to play Devil's advoca,
Speaker 2: open Ai likely added the routing model as a means
Speaker 2: of creating a more sophisticated output for a user, and
Speaker 2: I imagine with the intention of cost saving. Then again,
Speaker 2: this might just be the thing it had to ship.
Speaker 2: After all, GPT five was meant to be the next
Speaker 2: great leap in AI, and the pressure was on to
Speaker 2: get it out the door by creating a system that
Speaker 2: depends on an extern and or routing model, likely another LM.
Speaker 2: In this case, open ai has removed the ability to
Speaker 2: cash the hidden instructions that dictate the how the models
Speaker 2: generate answers in chat GPT, creating massive infrastructural overhead. Worse still,
Speaker 2: this happens with every single turn as in message on
Speaker 2: Chat GPT five, regardless of the model you choose, creating
Speaker 2: endless infrastructural baggage with no real way out that only
Speaker 2: could pounds based on how complex the user's queries get
Speaker 2: or how much they change. They could be simple, but
Speaker 2: just going in different directions every time, could open ai
Speaker 2: make a better router? Sure? Does it have a good
Speaker 2: one today? No, every time you message CHATGBT as the
Speaker 2: potential to change model or tooling based on its own whims,
Speaker 2: each time requiring a fresh static prompt, and short of
Speaker 2: totally reworking the architecture of chat GPT five, there's no
Speaker 2: way to change this. And if it's an LLM choosing
Speaker 2: which model, I don't know, maybe it hallucinates just a guess.
Speaker 2: It doesn't even need to be the case where a
Speaker 2: user asks chet GPT five to think, and based on
Speaker 2: my test with GPT five, sometimes you can just ask
Speaker 2: it a forward question and it will think about it.
Speaker 2: For no apparent reason, open ai has created a product
Speaker 2: with latency issues and an overwhelmingly convoluted routing system that's
Speaker 2: already straining capacity, to the point that this announcement feels
Speaker 2: like open ai is walking away from its API entirely. This,
Speaker 2: as a reminder, is the thing that people use to
Speaker 2: incorporate open AI's models into their apps while also running
Speaker 2: set models on the infrastructure open Ai rants from Microsoft
Speaker 2: and call even at some point as well as Oracle,
Speaker 2: and this API thing is really weird by the way
Speaker 2: because these are new models, but Open Eyes really not
Speaker 2: talking about the models themselves that much. Unlike the GPT
Speaker 2: four rower announcement, which mentions the API in the first paragraph,
Speaker 2: the GPT five announcement has no reference to it and
Speaker 2: only has a single reference to developers at all when
Speaker 2: talking about coding. Some woman has already hinted that he
Speaker 2: intends to deprecate any new API demand, though I imagine
Speaker 2: it will let anyone who will pay for priority processing,
Speaker 2: which is essentially open eyes way to require minimum commitments
Speaker 2: and extra payments from API customers just so they never
Speaker 2: feel the bite of any compute shortages and throttling, which
Speaker 2: they absolutely will do to people that don't pay. Chat
Speaker 2: GPT five feels like the ultimate comeuppance for a company
Speaker 2: that has never been forced to build a product, choosing
Speaker 2: instead to bolt increasingly complex tools onto the side of
Speaker 2: models in the hopes that one will magically appear. Now,
Speaker 2: each and every feature of Chat GPT burns more money
Speaker 2: than it ever did before. Chat GPT five feels like
Speaker 2: a product that was rushed to market by a desperate
Speaker 2: company that had to get something out of the In
Speaker 2: simpler terms, here, it's actually really funny. When I worked
Speaker 2: this out, I chuckled. I chuckled vigorously. This is just
Speaker 2: a case where open ai has given chat gpt middle manager.
Speaker 2: But now I'm giving you the chance to open up
Speaker 2: your hearts and do something better. Open up your wallets too,
Speaker 2: and send money to a company that follows here, But
Speaker 2: hold my advertisements and we're back. Like every great middle manager,
Speaker 2: chat GPT five's rutter creates more work based on its
Speaker 2: own interpretation of what's going on, and has a separate
Speaker 2: large language model. I can't imagine it has a ton
Speaker 2: of training data available if I had to guess, and
Speaker 2: this is a guess by the way open ai has done,
Speaker 2: and we'll do a lot of fine tuning and reinforcement
Speaker 2: learning to make it work. Though, to give it a
Speaker 2: little grace, this is a new thing that it's doing,
Speaker 2: and it's doing sort of a huge scale. The problems start,
Speaker 2: by the way, with the fact that chat GPT five
Speaker 2: is taking the user's initial prompt and then deciding which
Speaker 2: model to use, unlike previous models, which sent your prompt
Speaker 2: directly to the model along with the static prompt which
Speaker 2: was cashed and came first. An important feature in how
Speaker 2: these models, limit tokenburn. Open ai starts with a router
Speaker 2: model that makes takes what you ask and gives its
Speaker 2: chat GPT and tags it based on what kind of
Speaker 2: thing your question might need. The thing might be a tool,
Speaker 2: such as whether it has to do a web search
Speaker 2: to spit out the thing at the end, a reasoning model,
Speaker 2: whether it needs to use a coding language, and so
Speaker 2: on and so forth. Once chat GPT has bounced your
Speaker 2: query across various models, burn and compute along the way,
Speaker 2: it then pushes it towards the chat portion of the generation.
Speaker 2: And each time you ask chat GPT a question or
Speaker 2: to do something and you specialized static prompt is generated,
Speaker 2: sometimes several make it impossible to cash them in advance.
Speaker 2: In simpler terms, each time you message it, chat GPT
Speaker 2: is to dump all cased information and instructions for what
Speaker 2: you need to do and reload it with each prompt.
Speaker 2: Now here's some examples of what chat GPT five has
Speaker 2: to reload every single time you prompt him whether or
Speaker 2: not to use a browser or search the internet, and
Speaker 2: under what conditions to do so, because they will change
Speaker 2: with each prompt. How to approach a particular problem based
Speaker 2: on what the user asked, including any specific ways you
Speaker 2: meant to answer, tone, brevity, and so on based on
Speaker 2: their request, specifics around how it might use, say open
Speaker 2: ais code interpreter, such as the usage rules for running
Speaker 2: a Python script, or how you want the code's output,
Speaker 2: which again will be different based on each prompt. And
Speaker 2: you can even say, do it in the exactly the
Speaker 2: same way, and because it's a large language model, it
Speaker 2: may hallucinate something different every single goddamn time you prompt
Speaker 2: chat GPT five it has to do this. Worse still,
Speaker 2: a particular conversation can involve you using multiple different models
Speaker 2: and tools, requiring you with each and every prompt, having
Speaker 2: to inject a different static prompt for each component that
Speaker 2: chat GPT five uses. And you can't catch the static
Speaker 2: prompt before the user's intent because if you did that,
Speaker 2: it might send an instruction to a model that doesn't
Speaker 2: make sense, such as telling a reasoning model to give
Speaker 2: a quick and simple line answer remini or nanomodel to
Speaker 2: do some sort of deep reasoning, which would create a
Speaker 2: crappy answer and burn tokens in the process. And this
Speaker 2: is all thanks to the complicated way that open ai
Speaker 2: insisted on building GPT five. Every single time you send
Speaker 2: something to chat, GPT can trigger it to use a
Speaker 2: different series of models audio vision, reasoning, each with their
Speaker 2: own instructions, static prompts, all while pulling different tools, each
Speaker 2: requiring their own instructions based on what you asked, and
Speaker 2: reasoning models even have different depths of reasoning. Unlike four
Speaker 2: to ZH, which is a multimodal model combining text, vision,
Speaker 2: and voice, GPT five is a ratking of open AI's
Speaker 2: models and tools that gets reborn every single time you
Speaker 2: ask it to do anything prompt It can prompt cash
Speaker 2: some things, but the core instructions not so much. But
Speaker 2: let's get a little more granular, because I know I've
Speaker 2: been quite repetitive, but this is detailed. So from what
Speaker 2: I've been told, there are either one or two models
Speaker 2: at work for the routing. I'm going to go with
Speaker 2: what I think is most likely based on the discussions
Speaker 2: I've had with people familiar with the architecture. I've heard
Speaker 2: the term orchestrator thrown around potential to potentially suggesting the
Speaker 2: router may be more omnipresent throughout the process, but I
Speaker 2: was unable to confirm its existence. Reach out of you
Speaker 2: here differently, I'll explain things as they were explained to me. Though.
Speaker 2: When a user sensor prompt, it goes through the Splitter leg,
Speaker 2: which decides to send the query on one of two paths.
Speaker 2: One is called the fast path, where a query is straightforward,
Speaker 2: such as a text only conversation that doesn't require any
Speaker 2: analysis or extra tools or thinking, a path where the
Speaker 2: query may require reasoning or more complex tools like codgeneration
Speaker 2: or access to web browser for research. To be clear,
Speaker 2: there are prompts where it may be split into multiple
Speaker 2: paths that trigger multiple models or tools, each requiring their
Speaker 2: own static instructions. From what I understand, the splitter model
Speaker 2: is a completely separate large language model, though we don't
Speaker 2: have a ton of details about it. I also, based
Speaker 2: on conversations I've had, think there's a chance there could
Speaker 2: be a separate model that sits above the splitter that
Speaker 2: does much lighter classification of how a query might be routed.
Speaker 2: So you ask it to do something, it might just
Speaker 2: go Okay, this looks like it needs a tool and
Speaker 2: going off. Why now? In any case, none of this
Speaker 2: can be cashed because all of this exists before inference,
Speaker 2: which is where, by the way, it's inference I've misstated
Speaker 2: in the past. Is like it inferring, meaning inference is
Speaker 2: everything that happens to get an output to you. So
Speaker 2: all of the stuff that's happening. And by the way,
Speaker 2: this is all a completely new cost that open ai
Speaker 2: has created. No one does this like this, it's so
Speaker 2: fucking stupid. But now we get to the chat leg.
Speaker 2: Now the open ai has added layers of extraction, it
Speaker 2: can begin cooking up the output, by which I mean
Speaker 2: do inference. The chat leg is where the pieces that
Speaker 2: the splitter model created are pulled together, each loaded into
Speaker 2: their with their respective static prompts based on what the
Speaker 2: user asked chat GPD five to do. Each piece of
Speaker 2: the model a tool to generate Python and an image
Speaker 2: generation tool a reasoning model. To generate an output has
Speaker 2: to process an entirely new static prompt and again that's
Speaker 2: every interaction. Remember, static prompts are effectively instruction. So the
Speaker 2: splitter model has told each piece of the pie how
Speaker 2: to act to create a particular output. As a result,
Speaker 2: much of this can't be cashed, creating more and more
Speaker 2: repetitious token bone response and mean to have to repeat
Speaker 2: this stuff so that you really get him. The upshot
Speaker 2: of the chat legs static prompt baggage is that you
Speaker 2: can do a little more here, at least in theory,
Speaker 2: because each component can be instructed separately, they can again,
Speaker 2: in theory, be made to give more individualized, specialized outputs,
Speaker 2: like creating an image with tags that is as I'll
Speaker 2: give you an example of very shortly generated using a
Speaker 2: specific reasoning model. I'm clutching it straws here. I don't
Speaker 2: really know if this's better, but I'm trying to be reasonable.
Speaker 2: I'm trying to be normal. Every day, I try and
Speaker 2: be normal. Previously, Open Eye's advantage was that a model
Speaker 2: like four to oh was a kind of a jack
Speaker 2: of all trades. But to get the benefits of chat
Speaker 2: GPT five and that's in air quotes, it's engaged a
Speaker 2: conductor model that can just make things more convoluted, even
Speaker 2: in the case of simple requests. Let me give you
Speaker 2: an example. You upload a chart of NFL player's stats
Speaker 2: and ask chat GPT to decide which is the best
Speaker 2: of the group and create an image to show the results.
Speaker 2: In GPT four oh, chat GPT would use one more
Speaker 2: and thus one static prompt to look at the image,
Speaker 2: decide which tools to use, and then how to format
Speaker 2: the response. You only needed one prompt, which was cased
Speaker 2: because one model can look at the stats for all
Speaker 2: the data and make the decisions and then use the
Speaker 2: image generation tool to make the final image. In GPT five,
Speaker 2: the chet GPT conductor model would see the stats, root
Speaker 2: it to a vision model requiring its own static prompt,
Speaker 2: then a separate text only reasoning model, one that has
Speaker 2: no ability to use tools, but it might be cheaper
Speaker 2: to get an answer from and also requires a static prompt,
Speaker 2: and that would then decide which players are best and
Speaker 2: then spit out an output, and then root it to
Speaker 2: a completely separate model that can generate texts to query
Speaker 2: the image tool again need a stag prompt for this
Speaker 2: to then generate the image. On top of all this
Speaker 2: onerous baggage lies another problem. The GPT five's various models
Speaker 2: are just more complex. By splitting out the component elements
Speaker 2: of what a model can do and allowing each model
Speaker 2: to have different levels of reasoning, even the cheaper ones
Speaker 2: like MIDI and nano open AI has created an endless
Speaker 2: combination of different reasons to have to make a brand
Speaker 2: new static prompt instruction automated by a router, a large
Speaker 2: language model that chooses what large language model to choose
Speaker 2: for a query. It is, if I'm honest, kind of funny.
Speaker 2: Reasoning models work when simply described by breaking up a
Speaker 2: prompt into component pieces, looking over them, and deciding what
Speaker 2: the best course of action might be. Chat GPT's router
Speaker 2: is effectively an abstraction higher breaking up the prompt into
Speaker 2: component pieces, then choosing different models for each of those pieces,
Speaker 2: which may in turn be broken up by a reasoning model.
Speaker 2: While I wouldn't say this is a hat on a
Speaker 2: hat situation, it is at this point unclear what exactly
Speaker 2: the benefits of chat GPT five's new architecture are, less hallucinations,
Speaker 2: better answers. Based on what I've been told, this was
Speaker 2: a decision made to increase the model's performance, what I
Speaker 2: can say is that this very likely increased open ayes
Speaker 2: overhead at a time when it needs to do the
Speaker 2: exact opposite. Even if chat GPT five pushes people towards
Speaker 2: cheaper models, it does so while guaranteeing extra costs and
Speaker 2: latency and whatever signals it may learn as people use.
Speaker 2: This will have to create significant benefits massive one hundred
Speaker 2: percent plus game for it to be anything close to worthwhile.
Speaker 2: While open ai is rude to may be smart in
Speaker 2: terms of nuance of how it might answer a query,
Speaker 2: and even that I question it most decidedly, is not
Speaker 2: more efficient and may have actually increased the burn rate
Speaker 2: for a company that will lose as much as eight
Speaker 2: billion dollars this year, and I think that number might
Speaker 2: be low too. Yet what I'm left with in writing
Speaker 2: this script is how wasteful all of this is. Open Ai,
Speaker 2: a company that is already incinerated upwards of fifteen billion
Speaker 2: dollars in the last two years, has chosen to create
Speaker 2: a less efficient way of doing business as a means
Speaker 2: of eking out and monest the best performance improvements. It
Speaker 2: just sucks. In our own lives, we're continually pushed and
Speaker 2: pressured and punished if we get into debt, judged by
Speaker 2: our peers and our parents, if we spend our money recklessly,
Speaker 2: and if we're too reckless, we find ourselves less likely
Speaker 2: to receive anything from credit to housing. Companies like open
Speaker 2: Ai live by a different set of standards. Some Mormon
Speaker 2: intends to lose more than forty four billion dollars by
Speaker 2: the end of twenty twenty eight on open Ai, and
Speaker 2: graciously told CNBC, like Lord Farquad, he was willing to
Speaker 2: run at a loss for a long time where he
Speaker 2: was treated like he was this smart, reasonable decision maker
Speaker 2: rather than someone that needs to rein in their horrendous
Speaker 2: spending habits and be more mindful. The ultra rich are
Speaker 2: rewarded far more for their errant spending habits than we
Speaker 2: ever are for any thrifty inness or austerity measures we make,
Speaker 2: and none of us are afforded the level of grace
Speaker 2: that Clammy sam Altman has been and has been feels appropriate.
Speaker 2: Chat GPT five is an engineering nightmare, a phenomenally silly
Speaker 2: and desperate attempt to duce what remains of the dying
Speaker 2: innovation and excitement within the walls of open Ai. It's
Speaker 2: not November twenty twenty two anymore. And let's be honest,
Speaker 2: there really hasn't been anything exciting or interesting out this
Speaker 2: company since GPT four. There's nothing exciting happening at this company.
Speaker 2: As many as seven hundred million people a week allegedly
Speaker 2: used chat GPT, but nobody can really say why. An
Speaker 2: open Ai, despite its massive popularity. Cannot seem to stop
Speaker 2: losing billions of dollars, and it can't seem to explain
Speaker 2: why that's necessary other than this shit's really expensive. Dude,
Speaker 2: Can anyone actually articulate a reason why we need to
Speaker 2: burn billions of dollars to do this? What are we doing?
Speaker 2: Why are we doing it? Has everybody just agreed to
Speaker 2: do this until it becomes a completely untenable Do we
Speaker 2: all yearn for the abyss so much that we can't
Speaker 2: find camaraderie and admitting we were wrong? Look at GPT five.
Speaker 2: This is, if you believe the hype, the best funded,
Speaker 2: best resourced company in the world, with the greatest mind
Speaker 2: and its helm and the greatest minds within its wars.
Speaker 2: And this is the best they've gone. A large language
Speaker 2: model that chooses which large language model will answer your question.
Speaker 2: G fucking wit, Sam Mortman sounds dandy, and how much
Speaker 2: better is this? You say, Oh, you can't really say
Speaker 2: fucking brilliant? Hey does it do anything new?
Speaker 3: No?
Speaker 2: Oh, what's that? It's actually our job to work out
Speaker 2: for ourselves. Thanks man, I love it. I love this shit.
Speaker 2: And if you're someone that is a hype merchant listening
Speaker 2: to this and you've done really well getting to the
Speaker 2: end of the third part. By the way, I respect you.
Speaker 2: I want you to email me and explain why they
Speaker 2: should be justified in burning billions of dollars if you
Speaker 2: tell me, if you tell me Aws, I will eat
Speaker 2: you alive. I mean that, does it? I mean that
Speaker 2: completely literally, I will unhinge my jaw. I'll eat you
Speaker 2: like Kirby and shit out of dance. I've said that
Speaker 2: one before, but I'm going with him in any case.
Speaker 2: This three parter has also really reminded me how ridiculous
Speaker 2: this is, how nonsensical things have become, and how much
Speaker 2: waste has been kind of justified, justified on this idea
Speaker 2: that this will become something by people that don't really
Speaker 2: know what it does today or might do in the future.
Speaker 2: None of this is going to end well, and not
Speaker 2: even the boosters seem to be having fun anymore. Everybody's
Speaker 2: just flating around waiting for it to end. Even Sam
Speaker 2: Ortman seems tired of it all. I know, I bloody
Speaker 2: well I am. Thank you for listening to Better Offline.
Speaker 3: The editor and composer of the Better Offline theme song
Speaker 3: is Metosowski. You can check out more of his music
Speaker 3: and audio projects at Mattasowski dot com m A T
Speaker 3: T O S O W s Ki dot com. You
Speaker 3: can email me at easy at better offline dot com
Speaker 3: or visit better offline dot com to find more podcast
Speaker 3: links and of course, my newsletter. I also really recommend
Speaker 3: you go to chat dot where's youreaed dot at to
Speaker 3: visit the discord, and go to our slash.
Speaker 2: Better Offline to check out I'll Reddit. Thank you so
Speaker 2: much for listening. Better Offline is a production of cool
Speaker 2: Zone Media. For more from cool Zone Media, visit our
Speaker 2: website cool Zonemedia dot com, or check us out on
Speaker 2: the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.