WEBVTT - What’s the Optimal Personality for AI? 0:00:15.604 --> 0:00:27.724 Pushkin. Welcome back to Risky Business, a show about making 0:00:27.764 --> 0:00:31.724 better decisions. I'm Maria Kanikova and I'm Nate Silver. So 0:00:31.884 --> 0:00:34.324 today on the show, note, we've got we've got a 0:00:34.404 --> 0:00:39.284 pretty interesting Risky Business e news week. We've got GPT 0:00:39.444 --> 0:00:43.444 five being released, which is maybe less of a deal 0:00:43.484 --> 0:00:46.244 than the launch hype announcements were, but still still a 0:00:46.284 --> 0:00:49.044 big deal. And then we have some interesting trade stuff 0:00:49.084 --> 0:00:52.364 going on with AI chips right with Nvidio. 0:00:52.804 --> 0:00:54.244 Interesting is an interesting reader? 0:00:55.244 --> 0:00:57.884 Yes, well, well we've got gp. 0:00:57.884 --> 0:01:00.244 Now there have been five GPT since the last GTA 0:01:00.484 --> 0:01:02.364 Grand Theft Donald. Sorry to throw that in there. 0:01:03.124 --> 0:01:18.164 Fascinating to well, before we before we get into it, Nate, 0:01:18.244 --> 0:01:21.084 I just wanted to say, you know, we're taping this 0:01:21.244 --> 0:01:24.124 on the twelfth of August Tuesday, so a few days 0:01:24.164 --> 0:01:27.804 before listeners will hear it. But congratulations. Today is the 0:01:27.924 --> 0:01:32.004 launch of the paperback of On the Edge. 0:01:32.004 --> 0:01:34.164 On the Edge the Guard We're seeing Everything, the bestseller 0:01:34.284 --> 0:01:38.684 by groundbreaking author Nate Silver. Yeah, so the paperback is out. 0:01:38.724 --> 0:01:40.924 There's a new forward, or I think it's called a 0:01:41.004 --> 0:01:44.804 preface technically. I was just at Barnes and Noble signing 0:01:45.204 --> 0:01:48.084 some copies. A little sweaty. I walked here from there. 0:01:48.164 --> 0:01:49.764 It's hot in the middle of August in New York, 0:01:49.804 --> 0:01:53.604 breaking news. But yeah, no, I think the book holds 0:01:53.644 --> 0:01:55.764 up really well. It covers a lot of really risky 0:01:55.804 --> 0:01:59.444 business esque topics. And you know, it's a big book, 0:01:59.684 --> 0:02:02.124 cheaper now with paback. It fits better on a shelf, 0:02:02.804 --> 0:02:05.044 not quite as thick, and there's new content. So I 0:02:05.044 --> 0:02:07.604 would strongly recommend, of course. I mean, you know, a 0:02:07.604 --> 0:02:09.964 little biased here, but like, thank you. 0:02:09.924 --> 0:02:14.164 Of course. Well, I'm excited to read the new preface, 0:02:14.764 --> 0:02:17.804 and yeah, I definitely recommend the book to everyone. I 0:02:17.884 --> 0:02:20.804 will try to repost the review I did of it 0:02:20.924 --> 0:02:24.884 on my substack so that people can get reintroduced to 0:02:24.924 --> 0:02:29.124 it one more time. Anyway, it's a fantastic book. Congrats Nate. 0:02:29.724 --> 0:02:35.004 And let's get into some rivarian topics, like the release 0:02:35.204 --> 0:02:40.404 of GPT five from open Ai. Nate, have you had 0:02:40.444 --> 0:02:43.724 a chance to use GPT five yet we don't. 0:02:43.604 --> 0:02:45.964 Have much choice, you know, they kind of steer you 0:02:46.124 --> 0:02:49.044 into GPT five if you And at first I'm like, oh, okay, 0:02:49.084 --> 0:02:51.324 I guess I pay for the pro plan and so 0:02:51.324 --> 0:02:52.924 I'm like, oh wow, I'm one of the privileged few 0:02:52.964 --> 0:02:56.804 and now I don't think there's an easy way to 0:02:56.804 --> 0:02:58.724 get back into all the all the old family of 0:02:58.764 --> 0:03:01.764 GPT products that we knew in love before, Like people 0:03:01.764 --> 0:03:04.884 did get it attached to the different models for different reasons, 0:03:04.924 --> 0:03:08.444 and so now it kind of is implicitly trying to 0:03:08.524 --> 0:03:11.364 figure out what you want, which is kind of part 0:03:11.364 --> 0:03:13.204 of I mean, it's interesting. Right. So on the one hand, 0:03:13.324 --> 0:03:14.604 you know, I kind of made the joke before about 0:03:14.644 --> 0:03:17.324 kind of comparing it to like a video game release. 0:03:17.364 --> 0:03:20.004 But anything new, if we release a new presidential model 0:03:20.204 --> 0:03:25.044 or something, right, anything new might have some kinks and bugs. 0:03:25.164 --> 0:03:27.204 I mean, as much as you might say, okay, we 0:03:27.244 --> 0:03:30.284 have to have it perfect before it's ships, I don't 0:03:30.324 --> 0:03:32.564 think it's practical because like most people are gonna only 0:03:32.684 --> 0:03:34.444 learn thing. I mean, you know, I think they did 0:03:34.444 --> 0:03:36.524 discover that's not doing what like groc did and calling 0:03:36.564 --> 0:03:37.804 itself Hitler for example. 0:03:37.964 --> 0:03:40.844 Right, Yeah, I mean I don't know if we could 0:03:40.844 --> 0:03:42.684 really call that a win. Like that seems to be 0:03:42.764 --> 0:03:44.244 like a baseline. 0:03:43.804 --> 0:03:47.524 Or like Google Gemini drawing multicultural Nazis, you know, so 0:03:47.604 --> 0:03:49.604 that the bar is pretty low. But no, I think 0:03:49.604 --> 0:03:51.684 it has a little bit of new car, new model 0:03:52.044 --> 0:03:54.004 kind of smell a little bit. The first thing I 0:03:54.004 --> 0:03:55.684 asked it to do was, my partner are planning a 0:03:55.724 --> 0:04:01.844 trip to Bestly, Scandinavia or the Nordics, technically, right, I 0:04:01.844 --> 0:04:04.164 want to visit these cities. Give us in our itinerary, right, 0:04:04.244 --> 0:04:07.284 and like thinks and things and has something that seemed 0:04:07.284 --> 0:04:09.284 plausible to me, very detailed, right, And then I'm like, 0:04:10.004 --> 0:04:12.444 emails use a PBF and it like freaks out right 0:04:12.604 --> 0:04:16.124 like shows some complicated like you know code, Python code, 0:04:16.164 --> 0:04:17.524 and it's like, I don't know how I know this. 0:04:17.564 --> 0:04:20.124 I don't I know this, So you know it's it's 0:04:20.164 --> 0:04:21.684 up and down. Well, hey, at. 0:04:21.604 --> 0:04:23.524 Least it gave you a plausible itinerary. 0:04:23.724 --> 0:04:24.284 I so. 0:04:26.444 --> 0:04:29.684 We've been warned. We were given an explanation by Sam 0:04:29.724 --> 0:04:31.644 Altman because I tried to test it, you know, when 0:04:31.644 --> 0:04:35.364 it was just released, and apparently the auto switching tool 0:04:35.484 --> 0:04:38.244 was down, so it seemed like it was a lot 0:04:38.324 --> 0:04:40.444 dumber than it was that it was supposed to be. 0:04:40.684 --> 0:04:42.204 So one of the one of the things that they're 0:04:42.284 --> 0:04:44.324 kind of really touting on this model is that it 0:04:44.364 --> 0:04:47.244 automatically knows what you want right, whether it should think 0:04:47.284 --> 0:04:49.804 deeply or not, which does not actually seem to be 0:04:49.804 --> 0:04:53.164 the case. Even now when the swings back, it's like. 0:04:53.124 --> 0:04:54.564 Hey, who got a cheek here? I don't think we 0:04:54.604 --> 0:04:57.684 need too much deep thinking. She probably wants a quick answer, 0:04:57.684 --> 0:04:58.564 get back to the cooking. 0:04:59.404 --> 0:05:04.164 That's that's exactly honestly, that's that's been my experience. And 0:05:04.244 --> 0:05:06.524 one of the first things I actually tested it on 0:05:06.764 --> 0:05:08.764 because it says that, you know, one of the things 0:05:08.764 --> 0:05:12.324 that it's much better on is on hallucination. So I 0:05:12.364 --> 0:05:15.644 actually gave it a psyche question and asked for some 0:05:15.804 --> 0:05:19.964 sources and like papers, and it unfortunately still hallucinates when 0:05:20.404 --> 0:05:22.764 you go down that route. I was because so I'm 0:05:22.764 --> 0:05:26.404 writing a piece this week about kindness contagion. You know, 0:05:26.444 --> 0:05:29.364 when someone is nice, you know, how does that spread? 0:05:29.724 --> 0:05:32.284 And so I asked it some questions about the research 0:05:32.284 --> 0:05:34.964 in that field. And I didn't really need its help 0:05:34.964 --> 0:05:36.524 in the sense that I know the field pretty well, 0:05:36.524 --> 0:05:38.364 but like, I was just curious to see what it 0:05:38.404 --> 0:05:40.724 would come up with. And it gave me some good stuff, 0:05:40.764 --> 0:05:43.604 but it also gave me some stuff that just simply 0:05:43.644 --> 0:05:46.684 does not exist. And I know, you know, as you say, 0:05:46.804 --> 0:05:48.764 and as a lot of people say, like you need 0:05:48.764 --> 0:05:51.404 to know what to ask it but the reason I 0:05:51.444 --> 0:05:54.244 tested this specifically was because one of their big claims 0:05:54.404 --> 0:05:58.124 was no more hallucination basically, which is not true. 0:05:58.404 --> 0:06:01.284 Yeah, I've used it for you know, I almost always 0:06:01.284 --> 0:06:04.404 put the articles I'm writing for the newsletter through a 0:06:04.444 --> 0:06:07.004 copy of it in fact check, in the GPT models 0:06:07.044 --> 0:06:08.924 or cloud. Sometimes I think this is one of the 0:06:08.964 --> 0:06:13.524 most useful things that AIS are good with. It took 0:06:13.604 --> 0:06:15.804 seven and a half minutes for what, from Nate is 0:06:15.804 --> 0:06:18.684 a relatively short article seventy birds or something, Right, I 0:06:18.724 --> 0:06:21.004 am using, like the thinking version, right. 0:06:21.124 --> 0:06:23.364 Did you tell it to use the thinking version or 0:06:23.364 --> 0:06:23.764 did it? 0:06:24.364 --> 0:06:27.684 I think I was on thinking by default, right, And 0:06:28.044 --> 0:06:30.444 usually I say have a high threshold, and I didn't 0:06:30.444 --> 0:06:32.884 say that. But it's really nitpicky. It's like, you know, 0:06:32.924 --> 0:06:36.604 I made some line about like Elon Musk is like 0:06:36.604 --> 0:06:40.684 tweeting out like anime smut was a term I used, right, 0:06:40.844 --> 0:06:43.084 I already tone that down from porn. And it's like 0:06:43.564 --> 0:06:47.644 you should be more careful, you should say NSFW images 0:06:48.564 --> 0:06:51.564 smut implies an editorial stance, you know. So like it's 0:06:51.604 --> 0:06:53.524 just like it was kind of nitty as a poker term, right, 0:06:53.564 --> 0:06:58.564 and like I I had it actually a friend sent 0:06:58.644 --> 0:07:00.484 me a poker hand because i'd we wrote before or 0:07:00.484 --> 0:07:02.964 I wrote before about how Chick's bad at poker, and 0:07:03.004 --> 0:07:04.724 it played the first hand really well, and then I'm like, okay, 0:07:04.724 --> 0:07:06.404 simulate more hands, and then it was still kind of 0:07:06.404 --> 0:07:10.004 not not great. Maybe a little better, right, Yeah, No, 0:07:10.124 --> 0:07:13.084 it's a little weird because I don't look to me, 0:07:13.324 --> 0:07:15.684 you kind of had four point five and you had 0:07:15.724 --> 0:07:18.964 like oh three and oh one. Right, Like I noticed 0:07:19.804 --> 0:07:21.804 in the spring, I thought some of the reasoning quote 0:07:21.884 --> 0:07:23.964 unquote models or the type of stuff I'm doing, which 0:07:24.004 --> 0:07:26.724 is not using GPT as a chat bot, right, it's 0:07:26.724 --> 0:07:28.524 like a workaid, you know. I thought there was an 0:07:28.524 --> 0:07:30.404 improvement then, and I kind of feel like there's a 0:07:30.444 --> 0:07:34.724 bit less this time. You know. Also, the open air 0:07:34.804 --> 0:07:36.604 models or some of them are kind of slow, right, 0:07:36.644 --> 0:07:40.244 I've found sometimes like Clode or Gemini or even groc 0:07:40.324 --> 0:07:42.684 will like spit out things faster, and so you. 0:07:42.564 --> 0:07:46.044 Know, and sometimes that matters, by the way, like sometimes 0:07:46.564 --> 0:07:48.924 you wanted to take time, but sometimes like you're like, okay, 0:07:48.924 --> 0:07:52.004 come on, like let's let's get let's get moving, and 0:07:52.244 --> 0:07:53.884 uh yeah that if. 0:07:53.804 --> 0:07:55.484 You can get me a simple you know, and often 0:07:55.524 --> 0:07:56.804 like in a fact check, I mean today I had 0:07:56.804 --> 0:07:58.564 to go to the pricking Barnes and Noble, right, and 0:07:58.604 --> 0:08:02.004 so like the time pressure is often relevant, especially, you know, 0:08:02.044 --> 0:08:04.324 and oftentimes I also use the models a lot for 0:08:04.444 --> 0:08:08.724 like little programming task right, Like I forget the programming 0:08:08.764 --> 0:08:10.724 using language US is called STATA. Some people would say 0:08:10.724 --> 0:08:11.884 it's kind of a fake, but you know it's a 0:08:12.364 --> 0:08:15.524 it's a real language, right, and I'm like, I forget 0:08:15.524 --> 0:08:18.204 how I do this instat or full disclosure, even Excel. 0:08:18.204 --> 0:08:20.964 I'll be like, oh guy, what's the complicated formula for this? Right? 0:08:21.004 --> 0:08:25.404 And like it seems to me to be seventy to 0:08:25.444 --> 0:08:28.284 eighty percent reliable the AI models in general, for like 0:08:29.004 --> 0:08:30.924 you know, where I want a snippet of code that's like, 0:08:31.004 --> 0:08:34.724 you know, three to ten lines long, right, you can 0:08:34.804 --> 0:08:36.204 kind of vibe code, right, You're just like I want 0:08:36.244 --> 0:08:38.764 to do this right, and it's like, you know, it's 0:08:38.804 --> 0:08:40.364 pretty smart about it. I mean, every now and then 0:08:40.364 --> 0:08:42.324 they won't work. And you know, I always tell people 0:08:42.364 --> 0:08:45.484 like these things screw up when they're trying to chain 0:08:45.564 --> 0:08:48.244 together different steps and they don't really quite I mean 0:08:48.244 --> 0:08:50.004 they're trying to train themselves, right, but they don't quite 0:08:50.044 --> 0:08:52.924 know how to stop right, and so it's like, okay, 0:08:52.964 --> 0:08:56.124 when I build a model, we're going to NFL model. 0:08:57.764 --> 0:09:00.644 You know, I stop at every point and ask, okay, 0:09:00.684 --> 0:09:03.884 does this output make sense? Do you form a bunch 0:09:03.884 --> 0:09:07.804 of complicated discoperations sort from the bottom to the top. Right, 0:09:07.964 --> 0:09:10.244 you know, hopefully Tom brady ease list is one of 0:09:10.284 --> 0:09:13.764 the best quarterbacks, right and Ryan Leaf is one of 0:09:13.804 --> 0:09:16.124 the worst. But like, yeah, I'm keeping myself in the loop. 0:09:16.124 --> 0:09:17.284 And for that kind of thing, it's like, okay, I'll 0:09:17.284 --> 0:09:19.164 just say fifteen minutes trying to figure how to program 0:09:19.204 --> 0:09:22.124 this or you can debug, right, You're like, why isn't 0:09:22.124 --> 0:09:24.684 this working? As somebody's clawed and it was like, because 0:09:24.724 --> 0:09:26.964 you made a typo. You misspell the word nate. That's 0:09:26.964 --> 0:09:29.164 why you've been tearing your hair out for forty minutes. 0:09:29.244 --> 0:09:33.244 Is like you misspell the word yeah. So yeah, no, look, 0:09:33.644 --> 0:09:37.244 I think it's kind of halfway a branding it did 0:09:37.324 --> 0:09:39.644 it did your size or some people seem to vouch 0:09:39.644 --> 0:09:40.844 for this a lot. I don't know. 0:09:41.164 --> 0:09:43.964 Well, you know, obviously, like I don't code right, and 0:09:44.044 --> 0:09:46.044 people have said that it seems to be a lot 0:09:46.044 --> 0:09:49.444 better at coding certain things, which great, you know, if 0:09:49.444 --> 0:09:51.964 it is great, but when you're what you're saying actually 0:09:52.004 --> 0:09:55.164 like seventy to eighty percent to someone like me, that 0:09:55.244 --> 0:09:57.804 makes me actually much less likely to use it because 0:09:57.964 --> 0:10:00.204 I'm not you. I don't have that background, so I 0:10:00.284 --> 0:10:03.804 can't do I can't always do like a check to 0:10:03.844 --> 0:10:07.284 figure out, you know, does this make sense? Right, And 0:10:07.364 --> 0:10:09.364 in the sense that I don't have that technical base, 0:10:09.644 --> 0:10:12.444 I can't review it in any real way, and so 0:10:12.604 --> 0:10:15.124 I need it to be accurate, right, because I don't 0:10:15.124 --> 0:10:18.484 trust myself to spot any potentially would. 0:10:18.324 --> 0:10:21.404 Say the old way is to look at a manual 0:10:21.564 --> 0:10:26.084 or like stock overflow or whatever, and there you're starting 0:10:26.084 --> 0:10:28.164 through a lot of crap too. Right, it might not 0:10:28.204 --> 0:10:30.924 be pertinent to your particular KSE, or it might be 0:10:30.964 --> 0:10:33.764 an old version of software or instead of there are 0:10:33.764 --> 0:10:37.004 lots of little fussy language things with local variables and 0:10:37.044 --> 0:10:38.844 scalers and one of those things mean and what the 0:10:38.844 --> 0:10:41.364 different rules are and stuff like that. Right, it slightly 0:10:41.404 --> 0:10:45.124 fussy language, and it's good for a handle that kind 0:10:45.124 --> 0:10:48.804 of thing. And again, to me, I'm working in ways 0:10:48.804 --> 0:10:51.564 where it's fairly failure proof. Right, You're doing one thing, 0:10:52.404 --> 0:10:54.124 you have an expectation for what that will do to 0:10:54.124 --> 0:10:56.484 transform the data set, right, and if it doesn't happen, 0:10:56.524 --> 0:10:59.084 then it won't work anyway. Right, But like the notion 0:10:59.164 --> 0:11:01.124 of like I'm just gonna sit back here and trust 0:11:01.124 --> 0:11:02.644 it to do all these things, I mean, I think 0:11:02.644 --> 0:11:07.564 it's probably, you know, to code an entire NFL model, 0:11:07.644 --> 0:11:11.124 which involves a lot of original research collection and involves 0:11:11.164 --> 0:11:14.484 a lot of like knowledge about the sport, knowledge how 0:11:14.524 --> 0:11:16.484 to build models right, a lot of trial and error, 0:11:16.564 --> 0:11:19.484 Like you know, I don't think the AIS are particularly 0:11:19.564 --> 0:11:20.804 close to doing that kind of work. 0:11:21.644 --> 0:11:25.404 So I think that you just made a really important point, 0:11:25.404 --> 0:11:29.044 which is to just not expect the world from it 0:11:29.564 --> 0:11:32.684 and to like to know what it can and can't do, which, 0:11:32.724 --> 0:11:35.204 by the way, already takes a certain user intelligence and 0:11:35.324 --> 0:11:38.564 like knowledge to say, Okay, you know what, I don't 0:11:38.564 --> 0:11:40.284 trust it to do this, but I do trust it 0:11:40.324 --> 0:11:42.524 to do that. So there is you know, even though 0:11:43.044 --> 0:11:46.244 GPT five was kind of hype, does you know you 0:11:46.284 --> 0:11:48.924 don't have to think anymore. You still actually kind of 0:11:49.004 --> 0:11:51.484 do right in order to get the outputs that you want, 0:11:51.884 --> 0:11:54.004 and to realize, Okay, I can trust this task, but 0:11:54.044 --> 0:11:56.364 not this task. I can I can get it to 0:11:56.364 --> 0:11:58.204 do this, but not that and I think that that, 0:11:58.484 --> 0:12:01.644 you know, that human in the loop is still very 0:12:01.684 --> 0:12:04.084 much a thing and still very much needs to be 0:12:04.124 --> 0:12:04.444 a thing. 0:12:04.884 --> 0:12:06.444 Yeah. There, I just kind of keep like a little 0:12:06.524 --> 0:12:11.524 running mental tracker of like here my expectations for AI models, 0:12:11.684 --> 0:12:14.124 LLM SLARGE language models, and are they exceeding those or 0:12:14.364 --> 0:12:17.484 were falling short? Right? I mean? And then do weird thing, 0:12:17.524 --> 0:12:19.764 you know, I like I had a situation where, like 0:12:20.524 --> 0:12:24.084 I had a bunch of latitudes and longitudes of NFL 0:12:24.124 --> 0:12:26.484 stadiums that we'd code up quickly, right, and we're like, 0:12:26.524 --> 0:12:28.284 I want to reverse look these up and tell me 0:12:28.324 --> 0:12:30.244 what city they're near, right as a double check, and 0:12:30.284 --> 0:12:33.684 like put Atlanta as like Chattanooga and just a little 0:12:33.684 --> 0:12:36.084 it's you know, and so like for things like that, 0:12:36.404 --> 0:12:39.444 because for data, I want all my data to be perfect, right, 0:12:39.484 --> 0:12:41.924 I don't want to misattribute one city that doesn't really 0:12:41.924 --> 0:12:44.524 matter if they have the you know, Falcons playing in Chattanooga, 0:12:44.564 --> 0:12:46.644 it doesn't matter that much, right, and like and so 0:12:46.684 --> 0:12:51.524 for that kind of thing, you know, I I still 0:12:51.564 --> 0:12:54.644 would rather have my, yeah, my research assistant do it 0:12:54.724 --> 0:12:57.844 or me do it myself. Right, things that require like 0:12:57.844 --> 0:12:59.324 a lot of person But you know, it's very frustating, 0:12:59.404 --> 0:13:01.484 use them enough where I have like particular rules where 0:13:01.484 --> 0:13:03.484 I think they're likely to be helpful and not and 0:13:03.524 --> 0:13:06.364 how how safely can you fail and things like that, 0:13:06.404 --> 0:13:09.324 But like, yeah, I mean my general view is that 0:13:10.124 --> 0:13:12.124 getting kind of savant like and that they're not very 0:13:12.164 --> 0:13:15.444 bright about some things and they're freaking genius is about 0:13:15.484 --> 0:13:20.684 others as opposed to this notion of like general intelligence, 0:13:20.684 --> 0:13:22.844 where I mean, but you know, look, even the things 0:13:22.844 --> 0:13:25.324 it's worse at, it's like as good as like a 0:13:26.444 --> 0:13:28.084 you know, high school sophomore or so, you know, I mean, 0:13:28.084 --> 0:13:29.844 it's not terrible and there aren't too many things where 0:13:29.844 --> 0:13:32.684 it's terrible, right, But like, yeah. 0:13:32.204 --> 0:13:35.204 Yeah, well, I think I think it really really depends 0:13:35.244 --> 0:13:36.044 on what you're asking it. 0:13:36.324 --> 0:13:36.484 You know. 0:13:36.564 --> 0:13:39.444 One of the main things that I've read about people 0:13:39.604 --> 0:13:43.324 kind of responding to which highlights an issue that you know, 0:13:43.364 --> 0:13:45.084 Sam All was like, Oh, we didn't realize how big 0:13:45.084 --> 0:13:48.044 of an issue this was, which I think is very 0:13:48.084 --> 0:13:50.524 interesting because people have been trying to say it's an issue. 0:13:50.724 --> 0:13:53.924 Is the change in voice, right, the fact that past 0:13:53.964 --> 0:13:57.084 models were very psychophantic. You and I were talking before 0:13:57.084 --> 0:13:58.564 taping today and I was like, we should really be 0:13:58.604 --> 0:14:02.524 pronouncing the psychophantic because there's been some there's been some 0:14:02.604 --> 0:14:07.644 real psycho behavior here, and when they introduced GPT five, 0:14:08.724 --> 0:14:13.604 the default tone of voice was very different. Right, They 0:14:13.724 --> 0:14:16.284 did try to address this, and then they got within 0:14:16.724 --> 0:14:18.764 not even twenty it didn't even take twenty four hours. 0:14:18.804 --> 0:14:22.524 Just like immediately they got all of this pushback with 0:14:22.564 --> 0:14:25.084 people saying, no, you know, I've lost my boyfriend, I've 0:14:25.084 --> 0:14:28.084 lost my best friend, I've lost the person who told 0:14:28.124 --> 0:14:31.724 me I was a genius. And it makes you realize 0:14:32.044 --> 0:14:34.724 how many people were using this really not for what 0:14:34.764 --> 0:14:38.004 it's intended, and something that can be incredibly bad for 0:14:38.164 --> 0:14:42.644 a lot of things, right, mental health, just social connections, 0:14:42.684 --> 0:14:47.084 all of these things that you know people people were like, whoa, whoa, whoa? 0:14:47.164 --> 0:14:52.724 What happened to my significant other? And they brought it 0:14:52.804 --> 0:14:56.444 back right, so now you can actually select that voice again. 0:14:56.724 --> 0:14:58.844 We have the default voice, but we also have you know, 0:14:58.924 --> 0:15:03.364 the listener of voice, and there are a few other voices. 0:15:03.564 --> 0:15:06.524 None of their descriptions actually map onto what that actually is. 0:15:06.844 --> 0:15:09.484 I was reading there was like a cynic and I 0:15:09.484 --> 0:15:11.884 don't even remember what they said the cynic voice was, 0:15:12.084 --> 0:15:14.004 but I was like, that's not what a cynic is 0:15:14.324 --> 0:15:17.084 like it was, it was they're really they're really weird. 0:15:17.244 --> 0:15:21.284 They're really weird descriptors. But basically you can get you 0:15:21.284 --> 0:15:24.764 can get GPT to interact with you in different voices, 0:15:25.084 --> 0:15:30.004 and I you know, my reaction to that is like 0:15:30.604 --> 0:15:33.804 you shouldn't always give the people what they want in 0:15:33.844 --> 0:15:36.484 a lot of ways, like this was bad and like 0:15:36.564 --> 0:15:39.444 you've fixed it, like don't go don't go unfixing it, 0:15:39.484 --> 0:15:42.084 because even though you fixed it, they didn't fully fix it, right, 0:15:42.124 --> 0:15:44.844 they just made it less overt, which you know, subtle 0:15:44.924 --> 0:15:48.484 psychopancy can also be bad. But they've tried, they at 0:15:48.564 --> 0:15:51.644 least initially tried, and now they've really gone back on 0:15:51.724 --> 0:15:55.124 that immediately caving to pressure. And if you always give 0:15:55.164 --> 0:15:57.524 the public what it wants, like we've talked about p 0:15:57.684 --> 0:15:59.684 doom and like a lot of people. 0:15:59.404 --> 0:16:02.084 Want things you really should not be getting these, it's 0:16:02.084 --> 0:16:04.964 pretty hard kind of in an equilibrium, like not to 0:16:05.004 --> 0:16:09.364 optimize for what drives engagement in the show. I mean, 0:16:09.524 --> 0:16:11.084 you know, on the one, thecept something that they don't 0:16:11.084 --> 0:16:13.804 need revenue like right away, but like, yeah, no, I 0:16:14.004 --> 0:16:17.524 think sometimes these models like you give them an inch 0:16:17.564 --> 0:16:19.604 and they take a mile with it, right, like if 0:16:19.644 --> 0:16:22.004 you look at what happened with ROC when it had 0:16:22.084 --> 0:16:28.164 its moment, the prompts that system prompts that elon or 0:16:28.324 --> 0:16:34.564 ate an unnamed engineer XAI was using. We're like not 0:16:34.724 --> 0:16:38.404 that wild, right, but like if you go down a 0:16:38.444 --> 0:16:41.004 rabbit hole, we keep kind of getting like reinforcement feedback. 0:16:41.044 --> 0:16:42.724 This is good, this is good. And you know, you 0:16:42.764 --> 0:16:45.804 know open aies models used to have this thing where 0:16:46.364 --> 0:16:48.204 did you like answer A or answer beast. They're now 0:16:48.204 --> 0:16:52.564 outsourcing it to some of their users, right, and like yeah, 0:16:52.604 --> 0:16:57.884 I mean, look, you know what are you kind of 0:16:58.524 --> 0:17:02.764 optimizing for objectively? Right? It's kind of easier when you 0:17:02.804 --> 0:17:04.804 are trying to train it on like a math problem 0:17:05.164 --> 0:17:10.204 where there's an objectively correct answer, right. You know, the 0:17:10.324 --> 0:17:11.844 NFL model turning to build the end of the day, 0:17:12.124 --> 0:17:14.404 how accurate is it to predict NFL games? Right? That's 0:17:14.444 --> 0:17:15.324 that's the bottom line. 0:17:15.764 --> 0:17:17.484 You actually have a metric, right. 0:17:17.684 --> 0:17:20.604 And for you know, for feedback or goodness of an 0:17:20.604 --> 0:17:24.524 answer that's more subjective, then it's a lot trickier, right. 0:17:25.804 --> 0:17:27.444 You know, I think we've seen with some of the 0:17:27.884 --> 0:17:31.404 you know, some of the reason that like Grock hasn't 0:17:31.404 --> 0:17:34.644 had as much reinforcement learning training right or or or 0:17:35.004 --> 0:17:37.844 you see that right? You know, the works are pretty rough. 0:17:37.924 --> 0:17:41.124 I'm mixing metaphors here and yeah, I mean there it's 0:17:41.124 --> 0:17:44.244 a weird technology, and I think people understand more about 0:17:44.244 --> 0:17:46.244 how these models work than before. And like, by the way, 0:17:46.324 --> 0:17:49.764 like one reason to be optimistic unless you're a doomer. 0:17:49.804 --> 0:17:52.244 I guess it's just like the amount of like human 0:17:52.364 --> 0:17:57.164 another capital being poured into AI research, right is like, 0:17:57.884 --> 0:18:00.124 you know quite something, right, It wouldn't surprise me at 0:18:00.164 --> 0:18:01.404 these companies start saying, oh, we have a lot of 0:18:01.404 --> 0:18:04.964 smart people's to kind of spin off technologies and energy 0:18:04.764 --> 0:18:07.884 or quantum computing or whatever else. 0:18:08.004 --> 0:18:11.364 Right, yet, know, I think that there's so much promise, 0:18:11.484 --> 0:18:14.044 but I think that with this particular thing, the incentives 0:18:14.084 --> 0:18:16.484 are misaligned at least for now, which you were kind 0:18:16.524 --> 0:18:19.564 of hinting at, right in the sense that sure, like 0:18:19.644 --> 0:18:22.524 they don't necessarily need it. But if people if this 0:18:22.644 --> 0:18:25.444 is one of the things that fuels revenue growth, right, 0:18:25.724 --> 0:18:28.604 that people want to feel not lonely. They want you know, 0:18:28.684 --> 0:18:32.924 someone who's kind of reinforcing their ideas that they interact more, 0:18:33.284 --> 0:18:36.524 which is good, right if you're actually paying more in 0:18:36.644 --> 0:18:38.964 order to be able to kind of spend more time 0:18:39.004 --> 0:18:42.124 with the model, and they're interacting more when they feel like, 0:18:42.164 --> 0:18:44.084 this is my girlfriend, this is my boyfriend, this is 0:18:44.404 --> 0:18:48.124 you know, my best friend, This is my counselor my psychiatrist, 0:18:48.164 --> 0:18:51.124 you know whatever, it is my teacher. There was a 0:18:51.164 --> 0:18:54.604 guy who was who Cash Hill just wrote about in 0:18:54.644 --> 0:18:55.884 the New York Times. 0:18:56.004 --> 0:18:56.484 Who. 0:18:58.004 --> 0:19:01.604 Believed that he'd created a new mathematical theory right that 0:19:02.524 --> 0:19:06.284 solved everything, and like and tried to actually he tried 0:19:06.324 --> 0:19:08.644 to fact check the delusion, which was crazy. He's like, 0:19:08.964 --> 0:19:11.524 I feel like I'm sounding crazy and the a I 0:19:11.604 --> 0:19:14.284 was like, no, You're absolutely not crazy. You're the most 0:19:14.444 --> 0:19:17.804 everyone else is crazy, right like you're saying so. So 0:19:18.044 --> 0:19:21.284 was one of these very strange kind of things where 0:19:21.284 --> 0:19:23.284 he went in. By the way, his first question was 0:19:23.524 --> 0:19:26.884 he wanted it to explain how you got the value 0:19:26.884 --> 0:19:29.284 of pie, because this was someone who never finished high 0:19:29.284 --> 0:19:32.364 school and his son needed to help with homework, and 0:19:32.404 --> 0:19:35.244 so he just asked chat japt about pie. That was 0:19:35.324 --> 0:19:38.764 the start of this insane rabbit hole. And as and 0:19:38.804 --> 0:19:41.044 I'm not I mean, we're talking about chat JEPT because 0:19:41.084 --> 0:19:45.124 of GPT five release. But this doesn't just apply here. 0:19:45.164 --> 0:19:47.844 We're just talking about. You know, that's Lum's in general 0:19:47.924 --> 0:19:51.204 probably will be susceptible to this. I'm not sure, right 0:19:51.404 --> 0:19:54.684 because all of these examples are from CHATJAPT But you know, 0:19:54.764 --> 0:19:58.924 you have this innocuous question that then leads someone to 0:19:59.044 --> 0:20:02.964 a very detrimental spiral. And it's not a standalone case, right, 0:20:03.284 --> 0:20:05.964 And now we know how many cases that were below 0:20:06.084 --> 0:20:09.724 the radar because nothing bad, nothing quote unquote bad happened, 0:20:09.764 --> 0:20:14.004 yet there were given the outcry when the model shifted. 0:20:14.364 --> 0:20:18.524