WEBVTT - The Academics That Think ChatGPT Is BS 0:00:01.840 --> 0:00:02.880 All Zone Media. 0:00:04.040 --> 0:00:06.680 Hello and welcome to Better Offline. I'm your host ed 0:00:06.760 --> 0:00:22.280 z trun. In early June, three researchers from the University 0:00:22.280 --> 0:00:25.440 of Glasgow published a paper in the Ethics of Information 0:00:25.520 --> 0:00:29.159 and Technology Journal called chat GBT is Bullshit. And I 0:00:29.280 --> 0:00:31.680 just want to be clear. This is a great and 0:00:31.760 --> 0:00:35.000 thoroughly researched and well argued paper. This is not silly 0:00:35.080 --> 0:00:38.680 at all. It's actually great academia. And today I'm joined 0:00:38.680 --> 0:00:41.720 by the men who wrote it, academics Michael Townsend Hicks, 0:00:42.040 --> 0:00:45.160 James Humphries and Joe Slater, to talk about chat gbt's 0:00:45.200 --> 0:00:48.159 mediocrity and how it's not really built to represent the 0:00:48.159 --> 0:00:51.480 world at all. So, for the sake of argument, could 0:00:51.479 --> 0:00:53.360 you define bullshit for me? 0:00:54.600 --> 0:00:58.920 So you are bullshitting if you are speaking without caring 0:00:59.640 --> 0:01:02.560 about the truth of what you say. So normally, if 0:01:02.560 --> 0:01:06.000 I'm telling you staff about the world in a good case, 0:01:06.040 --> 0:01:08.640 I'll be telling you something that's true, and like trying 0:01:08.680 --> 0:01:11.280 to tell you something that's true. If I'm lying to you, 0:01:11.360 --> 0:01:14.000 I'll be knowingly telling you something that's false or something 0:01:14.080 --> 0:01:17.160 I think is false. I'm bullshitting. I just don't care. 0:01:17.680 --> 0:01:20.400 I'm trying to get you to believe me. I don't 0:01:20.440 --> 0:01:22.440 really care about whether what I say is true. I 0:01:22.520 --> 0:01:25.400 might not have any particular view on whether it's true. 0:01:25.280 --> 0:01:29.039 Or not right. And you define between like soft and 0:01:29.160 --> 0:01:31.520 hard bullshit? Can you also get into that as well? 0:01:31.840 --> 0:01:34.360 Can you also identify yourselves as well to. 0:01:34.440 --> 0:01:37.200 R Yeah, yeah, and Joe. So the soft bullshit hard 0:01:37.200 --> 0:01:41.039 bullshit distinction is a very serious and technical distinction, right, 0:01:42.319 --> 0:01:43.479 So he came up with. 0:01:43.440 --> 0:01:48.520 This because bullshit is in the technical philosophical sense. Comes 0:01:48.520 --> 0:01:52.880 from Harry Frankfurt, recently diseased but a really great philosopher. 0:01:52.720 --> 0:01:54.720 And he talks about the about. 0:01:54.520 --> 0:01:58.120 Bullshit that there is in a popular culture and just 0:01:58.200 --> 0:02:01.160 in general discourse these days. Some of the ways he 0:02:01.240 --> 0:02:04.280 talks about bullshit seemed to suggest that it needs to 0:02:04.320 --> 0:02:07.920 be accompanied by a sort of malign intention. 0:02:08.840 --> 0:02:10.919 I'm doing something kind of bad. 0:02:11.080 --> 0:02:15.200 I'm intending to mislead you about the enterprise of what 0:02:15.240 --> 0:02:15.639 I'm doing. 0:02:15.760 --> 0:02:18.120 Yeah, maybe about who you are or what you know. 0:02:18.200 --> 0:02:21.760 So you may be trying to portray yourself as someone 0:02:21.800 --> 0:02:25.799 who is knowledgeable about a particular subject. Maybe you're a 0:02:25.840 --> 0:02:28.280 student who showed up to class without doing the work. 0:02:28.800 --> 0:02:32.320 Maybe you're trying to portray yourself as someone who's virtuous 0:02:32.320 --> 0:02:34.640 in ways you're not. Maybe you're a politician who wants 0:02:34.639 --> 0:02:37.840 to seem like you care about your constituents, but actually 0:02:37.880 --> 0:02:41.480 you don't. So you're not trying to mislead somebody about 0:02:42.080 --> 0:02:46.000 what you're saying the content of your utterance. You're trying 0:02:46.040 --> 0:02:48.799 to mislead them instead about like why you're saying it's 0:02:50.120 --> 0:02:51.679 that's what we call hard bullshit. 0:02:51.840 --> 0:02:53.960 Yeah, it's one of the things Frankfort talked about. 0:02:54.200 --> 0:02:56.880 Yeah, so FRANKFOT doesn't make this hard bullshit soft bullshit 0:02:56.919 --> 0:02:59.720 the station, but we do because sometimes it seems like 0:02:59.760 --> 0:03:02.720 Frank for has this particular kind of intention in mind, 0:03:03.280 --> 0:03:06.240 but sometimes he's just a bit looser with it. And 0:03:06.280 --> 0:03:09.160 we want to say that chat GPT and other large 0:03:09.240 --> 0:03:12.799 language models they don't really have this intention to deceive 0:03:12.880 --> 0:03:15.160 because they're not people that don't have these intentions. They're 0:03:15.160 --> 0:03:17.680 not trying to mess with us in that kind of way, 0:03:18.040 --> 0:03:21.120 but they do lack this kind of caring about truth. 0:03:21.840 --> 0:03:24.520 Well, I'm James, by the way, I suppose we strictly 0:03:24.600 --> 0:03:27.320 don't want to say that they aren't hard bullshitters, and 0:03:27.360 --> 0:03:29.840 we just think if you don't think that large language 0:03:29.880 --> 0:03:31.640 models are sapient. If you don't think they're kind of 0:03:31.680 --> 0:03:35.080 minds in any important way, then they're not hard bullshitters. 0:03:35.080 --> 0:03:37.120 So I think in the paper we don't take a 0:03:37.120 --> 0:03:39.320 position on whether or not they are. We just say 0:03:39.800 --> 0:03:41.720 if they are, this is the way in which they are. 0:03:41.920 --> 0:03:45.080 But minimally they're soft bullshitters. So they're kind of soft bullshites, 0:03:45.120 --> 0:03:48.120 as Joe says, doesn't require that speakers attempting to deceive 0:03:48.160 --> 0:03:50.360 the audience about the nature of the enterprise. 0:03:50.560 --> 0:03:51.440 Hard bullshit does. 0:03:51.680 --> 0:03:54.040 So if it turns out that large language models are sapient, 0:03:54.080 --> 0:03:56.800 which they're definitely not, like that's just technoproperty. 0:03:57.160 --> 0:03:58.280 Yeah, that's nonsense. 0:03:58.400 --> 0:04:01.400 Yeah, But if they are, then they're bullshitters and minimally 0:04:01.440 --> 0:04:03.480 their soft bullshitters or their bullshit machines. 0:04:04.880 --> 0:04:07.880 So you also make the distinction in there the intention, 0:04:08.240 --> 0:04:10.920 so the very fabric of hard bullshit that you intentionally 0:04:10.960 --> 0:04:14.400 are bullshitting to someone, you kind of make this distinction 0:04:14.480 --> 0:04:19.000 that the intention of the designer and the involved prompting 0:04:19.160 --> 0:04:23.240 could make this hard bullshit. Because with a lot of 0:04:23.279 --> 0:04:27.000 these models and someone recently jail broke chat GPT and 0:04:27.040 --> 0:04:29.520 it listed all of the things that's prompted to do 0:04:29.880 --> 0:04:33.800 could prompting be considered intentional? That Sam Altman, CEO of 0:04:34.040 --> 0:04:37.000 open AI, could be intentionally bullshitting? I think it is. 0:04:37.320 --> 0:04:39.279 Yeah, this again, I think it's something I don't know 0:04:39.320 --> 0:04:41.680 what the kind of hive mind consensus on this is. 0:04:41.720 --> 0:04:44.080 I'm sort of sympathetic to the idea that if you 0:04:44.160 --> 0:04:48.479 take this kind of purposive or teleological attitude towards right, 0:04:48.920 --> 0:04:51.440 what an intention is is an effort to do something, 0:04:51.600 --> 0:04:54.320 then maybe they do have intentions. But again, I think 0:04:54.320 --> 0:04:55.720 in the paper we just sort of wanted I mean, 0:04:55.760 --> 0:04:59.080 it's a standard philosophical move, right, just sort of go, look, 0:04:59.080 --> 0:05:01.520 here's all this controversial stuff as we can make it. 0:05:01.760 --> 0:05:03.640 Now we can hit you with the really controversial shit 0:05:03.680 --> 0:05:05.799 that we wanted to get to. So in the paper 0:05:05.800 --> 0:05:08.359 we sort of deliberately when maybe you might think it 0:05:08.400 --> 0:05:10.760 has intentions for it this reason, we kind of have 0:05:10.839 --> 0:05:13.839 no judgment on this efficiently, I'm sympathetic to the sort. 0:05:13.640 --> 0:05:14.440 Of view that you're putting. 0:05:14.480 --> 0:05:15.920 I think you're kind of sympathetic this as well. 0:05:16.240 --> 0:05:17.160 Right, Yeah, so. 0:05:17.320 --> 0:05:19.080 I Mike, there are a few ways that you can 0:05:19.120 --> 0:05:21.960 think of CHATCHYBT as having intentions. 0:05:21.440 --> 0:05:23.920 I think, and we talked about a few of them. 0:05:23.960 --> 0:05:27.320 One is by thinking of the designers who created it 0:05:27.720 --> 0:05:30.280 as kind of imbuing it with intentions. So they created 0:05:30.320 --> 0:05:32.760 it for a purpose and to do a particular task, 0:05:33.520 --> 0:05:36.800 and that task is to make people think that it's 0:05:36.839 --> 0:05:38.160 a normal sounding person. 0:05:38.440 --> 0:05:38.640 Right. 0:05:38.680 --> 0:05:41.760 It's to make people when they have a conversation with it, 0:05:41.880 --> 0:05:45.760 not be able to distinguish between what it's outputting and 0:05:45.880 --> 0:05:47.680 what a normal human would say. 0:05:48.000 --> 0:05:48.200 Right. 0:05:48.640 --> 0:05:51.080 Right, And that kind. 0:05:50.880 --> 0:05:56.040 Of goal, if it amounts to an intention, is the 0:05:56.160 --> 0:05:59.040 kind of intention we think a bullshitter has. 0:05:59.120 --> 0:05:59.280 Right. 0:05:59.360 --> 0:06:01.920 It's not trying to deceive you about anything it's saying. 0:06:01.960 --> 0:06:04.120 It doesn't care whether what it's saying is true. That's 0:06:04.160 --> 0:06:06.280 not part of the goal. But the goal is to 0:06:06.279 --> 0:06:09.080 do is to make it seem as if it's something 0:06:09.080 --> 0:06:13.040 that it's not, like specifically a human interlocutor. And one 0:06:13.040 --> 0:06:16.240 source for that goal is the programmers who designed it. 0:06:16.680 --> 0:06:20.600 Another is the training method. So it was trained by 0:06:20.600 --> 0:06:23.760 being given sort of positive and negative feedback in order 0:06:23.839 --> 0:06:25.480 to achieve. 0:06:25.120 --> 0:06:28.680 A specific thing, right, And that specific thing is just 0:06:28.760 --> 0:06:29.520 sounding normal. 0:06:29.720 --> 0:06:31.640 And that's so similar to what our students are doing 0:06:31.640 --> 0:06:34.720 when they try to pretend that they read something they 0:06:34.760 --> 0:06:35.279 haven't read. 0:06:36.440 --> 0:06:39.479 There is something very collegiate about the way it bullshits though. 0:06:39.560 --> 0:06:42.159 It reminds me of when I was in college when 0:06:42.240 --> 0:06:45.280 Penn State and Aporistwyth two very different institutes. 0:06:45.040 --> 0:06:48.080 Right, both kind of in the middle of nowhere. 0:06:49.080 --> 0:06:52.120 Yeah, both very sad. But the one thing you saw 0:06:52.160 --> 0:06:54.960 with like students who were doing like B plus homework 0:06:55.040 --> 0:06:57.360 is they were using words they didn't really understand. They 0:06:57.360 --> 0:06:59.520 were putting things together in a way that realists, like 0:06:59.520 --> 0:07:03.440 the intro body conclusion, there was a certain formula behind it, 0:07:03.480 --> 0:07:06.279 and it's just it feels exactly like it. But that 0:07:06.360 --> 0:07:08.160 kind of brings me to my next question, which is, 0:07:08.480 --> 0:07:11.600 how did you decide to write this? What inspired this? 0:07:11.920 --> 0:07:12.000 Right? 0:07:12.040 --> 0:07:14.120 I would feel this because whenever Mike tells the story, 0:07:14.160 --> 0:07:16.800 you get sort of sanitized version. We were in the 0:07:16.800 --> 0:07:19.480 pub whining about student essays. Perfect, yeah, like. 0:07:19.720 --> 0:07:22.800 You were not long here, right, you're not long yeah. 0:07:22.800 --> 0:07:25.040 Long in post of a bunch of us, or went 0:07:25.080 --> 0:07:27.960 to the pub on emotional let's let's welcome our new 0:07:27.960 --> 0:07:30.720 members of staff sort of event, and inevitably with hat 0:07:30.720 --> 0:07:32.920 about two points we were all pissing in moaning and 0:07:33.440 --> 0:07:34.320 do you think it might have been. 0:07:34.200 --> 0:07:35.960 Neiled that kind of prompted this. No, I don't think 0:07:35.960 --> 0:07:36.360 he was the. 0:07:38.760 --> 0:07:40.560 We talked about it after, right, Yeah, in a case 0:07:40.600 --> 0:07:41.240 like sort of came up. 0:07:41.280 --> 0:07:43.040 We were talking about this sort of prevalence of chat 0:07:43.080 --> 0:07:46.560 GPT generated stuff and kind of what it said about 0:07:47.240 --> 0:07:50.240 how at least some students were kind of approaching the assessments, 0:07:50.320 --> 0:07:51.920 and you know, I forget who. Someone sort of just 0:07:51.960 --> 0:07:53.800 went off handly, yeah, it was all just Frank thirteen 0:07:53.840 --> 0:07:56.680 bullshit though, isn't it. And we sort of collectively went hello, 0:07:56.760 --> 0:07:58.880 because obviously, all having a background in plosphy, we've all 0:07:58.920 --> 0:08:01.200 read on bullshit. You know, we all went he we 0:08:01.280 --> 0:08:04.080 get to say bullshit and allegively serious academic work. So 0:08:04.160 --> 0:08:07.280 the start it was we've had this experience of having 0:08:07.280 --> 0:08:10.240 to read all of this kind of uncanny Valley stuff, 0:08:10.560 --> 0:08:12.960 and when prompted, we all went, oh, it really is 0:08:13.120 --> 0:08:15.000 like Frank thirty and bullshit in a way that we 0:08:15.000 --> 0:08:16.200 can probably get paper outselof. 0:08:16.320 --> 0:08:19.480 Yeah, And at that point I think we were in 0:08:19.520 --> 0:08:24.320 the department meetings talking about how to deal with chat 0:08:24.400 --> 0:08:27.640 GPT written papers, and there were discussions going on all 0:08:27.680 --> 0:08:29.800 over the university, including kind of in high levels and 0:08:29.800 --> 0:08:34.360 the administration to come up with a policy, and we 0:08:34.440 --> 0:08:38.960 specifically wanted a stronger policy because our university is very 0:08:38.960 --> 0:08:42.400 interested in cutting into technology, right, and so they wanted 0:08:42.440 --> 0:08:45.400 the students to have an experience using and figuring out 0:08:45.400 --> 0:08:48.559 how to use whatever the freshest technology is. 0:08:48.520 --> 0:08:52.719 Which as they should as they should, but not for essays, right, 0:08:52.880 --> 0:08:53.280 That's the. 0:08:53.240 --> 0:08:53.920 Worry we had. 0:08:54.040 --> 0:08:56.600 We thought, you know, if we're not very clear about this, 0:08:56.720 --> 0:08:58.360 the students will be using it in a way that 0:08:58.440 --> 0:09:01.360 will detract from the educational experience. 0:09:02.040 --> 0:09:05.400 Right. And at the same time, it was becoming more. 0:09:05.200 --> 0:09:08.760 Widely known how these machines work, like specifically how they're 0:09:08.800 --> 0:09:12.960 doing next token or next word prediction in order to 0:09:13.040 --> 0:09:17.079 come up with a digestible string of text. And when 0:09:17.080 --> 0:09:19.920 you know that that's how you're they're doing it, I mean, 0:09:21.040 --> 0:09:25.080 it seems so similar to what humans do when they 0:09:25.080 --> 0:09:28.000 have no idea what they're talking about. And so when 0:09:28.040 --> 0:09:30.760 we were talking about it just seemed like an obvious 0:09:30.800 --> 0:09:32.920 paper that someone was going to write, and we thought 0:09:32.960 --> 0:09:36.079 that a better bs. You know, eventually people are going 0:09:36.120 --> 0:09:38.000 to see this connection and it's. 0:09:37.800 --> 0:09:39.080 A great papa. 0:09:39.200 --> 0:09:40.160 I think it worked very well. 0:09:40.200 --> 0:09:41.880 I mean, I think that it's the kind of thing 0:09:41.920 --> 0:09:45.640 where when I pitch this to other philosophers, it doesn't 0:09:45.640 --> 0:09:48.840 take them very long to just agree to be like, ah, yes. 0:09:49.040 --> 0:09:49.680 That's right. 0:09:50.200 --> 0:09:54.080 It's an interesting philosophical concept as well, because the way 0:09:54.080 --> 0:09:56.319 that people look at chat GPT in large language models 0:09:56.400 --> 0:09:58.600 is very much machine do this. But when you think 0:09:58.640 --> 0:10:01.400 about it, there are the ways where people are giving 0:10:01.440 --> 0:10:03.600 it consciousness. I just sort of tweet just now as 0:10:03.640 --> 0:10:07.000 someone talking about saying please with every request. It's like, no, 0:10:07.160 --> 0:10:09.800 I will abuse the computer in whatever manner I see fit. 0:10:10.280 --> 0:10:15.520 But it's it's curious because I think more people need 0:10:15.559 --> 0:10:18.559 to be having conversations like this. And one particular thing 0:10:18.600 --> 0:10:20.120 I like that you said, I actually would love you 0:10:20.120 --> 0:10:22.280 to go into more detail, is you said, the chat 0:10:22.280 --> 0:10:25.680 GPT in large language buddles, they're not designed to represent 0:10:25.720 --> 0:10:28.440 the world at all. They're not lying or misrepresenting. They're 0:10:28.440 --> 0:10:30.160 not designed to do that. What do you mean by that? 0:10:30.520 --> 0:10:33.520 I mean kind of as background, I do philosophy of science, 0:10:33.640 --> 0:10:38.320 but my thoughts about something like CHATCHEPT are largely inspired 0:10:38.360 --> 0:10:40.280 by the fact that I also teach a class called 0:10:40.960 --> 0:10:42.800 Understanding Philosophy through Science Fiction. 0:10:43.080 --> 0:10:44.800 Oh and and now we like talk. 0:10:44.679 --> 0:10:48.679 About whether computers could be conscious and I don't know 0:10:48.720 --> 0:10:51.480 what you guys think. Actually I think they could, right, 0:10:52.880 --> 0:10:55.520 I just don't think this one is. And part of 0:10:55.520 --> 0:10:57.800 the reason I think they could but this one isn't 0:10:57.960 --> 0:11:00.120 is that I think that in order to sort of 0:11:00.160 --> 0:11:03.320 represent the world or have the kinds of things we 0:11:03.400 --> 0:11:07.480 have that are like beliefs, desires, thoughts that are about 0:11:07.640 --> 0:11:12.920 external things, you have to have internal states that are 0:11:12.960 --> 0:11:16.920 connected in some way to the external world. Usually causation. 0:11:17.080 --> 0:11:20.160 We're perceiving things, information is coming in. Then we've got 0:11:20.320 --> 0:11:23.440 some kind of state in our brain that's designed just 0:11:23.480 --> 0:11:25.839 to track these things in the external world. 0:11:26.080 --> 0:11:26.280 Right. 0:11:26.800 --> 0:11:29.480 That's a huge part of our cognitive lives. It is 0:11:29.679 --> 0:11:34.440 just tracking external world things and it's a very important 0:11:34.440 --> 0:11:36.839 part of childhood development when you figure out how to 0:11:36.880 --> 0:11:37.360 interact them. 0:11:37.520 --> 0:11:42.000 It's semiotics, right, Daniel John Aberistwyth told me semiotics it's 0:11:42.040 --> 0:11:43.000 like an reception of the world. 0:11:43.000 --> 0:11:45.959 And this is like theory of meaning stuff. So, yeah, 0:11:46.120 --> 0:11:51.599 semiotics is like theory of science. How is it that assign. 0:11:50.200 --> 0:11:52.480 Can be folks, the representation of the thing and the 0:11:52.520 --> 0:11:53.160 thing itself. 0:11:53.280 --> 0:11:55.080 That can happen, yeah, but not always. 0:11:55.080 --> 0:11:58.560 Sometimes it's just the representation of the thing and there's 0:11:58.840 --> 0:12:02.240 a lot of philosophy as at figuring out how brain 0:12:02.320 --> 0:12:05.840 states or words on a page can be about external 0:12:05.840 --> 0:12:09.480 world things. And a big part of it, at least 0:12:09.480 --> 0:12:13.240 from my perspective, has to do with tracking those things, 0:12:13.800 --> 0:12:16.959 keeping tabs on them, changing as a result of seeing 0:12:17.000 --> 0:12:20.880 differences in the external world. And chatch ept is not 0:12:20.960 --> 0:12:24.040 doing any of that, right, That's not what it's designed 0:12:24.040 --> 0:12:26.600 to do. It's taking in a lot of data once 0:12:27.400 --> 0:12:31.600 and then using that to respond to text. But it's 0:12:31.640 --> 0:12:37.559 not remembering individuals, tracking things in the world in any 0:12:37.640 --> 0:12:41.640 way perceiving things in the world. It's just forming a 0:12:41.720 --> 0:12:45.240 sort of statistical model of what people say. And that's 0:12:45.320 --> 0:12:50.880 kind of so divorced from what most thinking beings do. 0:12:51.600 --> 0:12:53.079 It's divorced from experience. 0:12:53.320 --> 0:12:54.800 Yeah. I mean, as far as I can tell, it 0:12:54.800 --> 0:12:56.679 doesn't have anything like experience. 0:12:56.960 --> 0:12:57.160 Yeah. 0:12:57.240 --> 0:12:59.400 And that's one of the things that this I thinks of. 0:12:59.640 --> 0:13:01.679 In one way it comes down to is that if 0:13:01.679 --> 0:13:03.680 you sort of post this sufficiently far, someone is going 0:13:03.760 --> 0:13:06.000 to go, ah, isn't this just biochauvinism, right, Like, aren't 0:13:06.000 --> 0:13:09.000 you just assuming that unless something runs on like meat, 0:13:09.360 --> 0:13:11.520 it can't be sentient? And this isn't something we get 0:13:11.559 --> 0:13:14.160 into into paper part because we didn't really think it 0:13:14.200 --> 0:13:18.360 was worth addressing. But the sorts of things that seem 0:13:18.520 --> 0:13:20.840 like they're like never mind consciousness, right, but it seemed 0:13:20.840 --> 0:13:22.640 to be necessary in order for something to be trying 0:13:22.960 --> 0:13:25.000 to track the world, or in order to be corresponding 0:13:25.040 --> 0:13:27.120 the world, or to form beliefs about the world. Chat 0:13:27.200 --> 0:13:29.959 GPT just doesn't seem to meet any of them. Kind Of, 0:13:30.040 --> 0:13:32.960 it's if it does turn out that it's sapient, then 0:13:33.000 --> 0:13:39.360 chat GPT has got some profoundly serious executive function disorders, right, sapient, Right, 0:13:39.360 --> 0:13:40.760 so we don't have to worry about it. But kind 0:13:40.760 --> 0:13:43.280 of it's not the case that we've got some blundering 0:13:43.320 --> 0:13:45.599 proto general intelligence that's trying to figure out how to 0:13:45.679 --> 0:13:47.480 represent the world. It's not trying to represent the world 0:13:47.480 --> 0:13:50.080 at all. It's utterances are designed to look as if 0:13:50.360 --> 0:13:52.040 it's trying to represent the world, and then we just 0:13:52.080 --> 0:13:53.320 go that that's just bullshit. 0:13:53.360 --> 0:13:54.920 This is a classic case of bullshit. 0:13:55.280 --> 0:13:57.680 Yeah, it seems to be making stuff up, but making 0:13:57.760 --> 0:13:59.880 stuff up doesn't even seem to describe it. It's just 0:14:00.280 --> 0:14:03.520 throwing shit at a wall very accurately, but not accurately enough. 0:14:03.600 --> 0:14:05.840 Yeah, it's got various guidelines that allow it to throw 0:14:05.880 --> 0:14:07.439 shit at the wall where a sort of reasonably high 0:14:07.440 --> 0:14:08.719 degree of Actually, no, you're right. I mean, one of 0:14:08.720 --> 0:14:10.800 the things that a human bullshitter could at least be 0:14:10.840 --> 0:14:13.320 characterized as doing is they'd have to try and kind 0:14:13.320 --> 0:14:15.800 of judge their audience, right, They'd have to try and 0:14:15.840 --> 0:14:18.440 make the bullshit plausible to the audience that they're speaking to, 0:14:18.720 --> 0:14:20.840 and chat GPT can't do that, right. All it can 0:14:20.880 --> 0:14:23.640 do is go on a statistically large enough model and 0:14:24.520 --> 0:14:25.239 it looks. 0:14:25.040 --> 0:14:27.080 Like Z follows why follows X? 0:14:27.800 --> 0:14:30.120 Right, it's not kind of got any does have any 0:14:30.160 --> 0:14:32.280 consciousness at all, of course, but it doesn't have any 0:14:32.480 --> 0:14:35.240 sensitivity to the sorts of things that people are in 0:14:35.280 --> 0:14:38.080 fact likely to find plausible. It just does a kind 0:14:38.080 --> 0:14:40.480 of brute number crunch. So it's more complicated than that, 0:14:40.520 --> 0:14:43.000 but I think it boies down effectively to kind of 0:14:43.080 --> 0:14:45.560 number crunching, right, kind of data and context. 0:14:45.240 --> 0:14:49.000 In which probabilistic planning of what planning, it doesn't plan. 0:14:49.240 --> 0:14:51.320 That's the thing. It's interesting. There are these words you 0:14:51.440 --> 0:14:53.400 use to describe things that, when you think about it, 0:14:53.440 --> 0:14:57.160 are not accurate. You can't say it plans or things. 0:14:57.960 --> 0:14:58.720 I think one thing. 0:14:58.840 --> 0:15:01.760 Between when we came up with the idea and when 0:15:01.800 --> 0:15:05.720 we finish writing the paper, we spent some time reading 0:15:05.760 --> 0:15:08.880 about how it works and how it represents language and 0:15:08.920 --> 0:15:10.560 what the statistical. 0:15:09.920 --> 0:15:10.840 Model is like. 0:15:11.160 --> 0:15:17.200 And I was maybe more impressed than James about that, 0:15:17.280 --> 0:15:20.760 because it is like doing things that are similar to 0:15:20.800 --> 0:15:23.760 what we do when we understand language. It does seem 0:15:23.800 --> 0:15:27.920 to kind of locate words in a meaning space right right, 0:15:28.360 --> 0:15:30.680 and connect them to other words, and you know, show 0:15:30.760 --> 0:15:33.480 similarity and meaning, and it also does seem to be 0:15:33.520 --> 0:15:38.560 able to in some way understand context. But we don't 0:15:38.560 --> 0:15:42.440 know how similar that is for a variety of reasons, 0:15:42.440 --> 0:15:45.520 but mostly because it's too big of a system and 0:15:45.560 --> 0:15:49.880 we can only kind of probe it. And it's trained indirectly, right, 0:15:50.000 --> 0:15:55.560 so it's not programmed by individuals. And even though that's 0:15:55.640 --> 0:15:58.240 kind of a very impressive model of language and meaning 0:15:58.480 --> 0:16:02.080 and may some ways be similar to what we do 0:16:02.160 --> 0:16:06.560 when we understand language, we're doing a lot more things 0:16:06.640 --> 0:16:10.920 like planning, things like tracking things in the world. Just 0:16:10.960 --> 0:16:13.960 having desires and representing the way you want the world 0:16:13.960 --> 0:16:16.960 to be and thereby generating goals doesn't seem to be 0:16:17.400 --> 0:16:18.880 something that it has anything any. 0:16:18.800 --> 0:16:19.880 Room for in its architects. 0:16:20.160 --> 0:16:21.480 This is something of the time of it's just goods 0:16:21.520 --> 0:16:23.200 me and you were talking about the kind of in 0:16:23.280 --> 0:16:25.040 some ways it learns language the same way. 0:16:24.880 --> 0:16:25.320 That we do. 0:16:25.480 --> 0:16:27.760 I mean, it's got no grasp of expleted in fixation, right, 0:16:27.800 --> 0:16:28.680 this is one of chumps here. 0:16:28.800 --> 0:16:31.040 What does that mean? Just for not for me? I 0:16:31.080 --> 0:16:31.680 definitely know. 0:16:32.160 --> 0:16:35.000 Yeah, yeah, of course if I give you the sentence 0:16:35.440 --> 0:16:37.840 that's completely crazy, man, and tell you to put the 0:16:37.840 --> 0:16:40.840 word fucking into that sentence, there's a number of ways 0:16:40.840 --> 0:16:42.800 in which any language speaker is going to do it 0:16:43.080 --> 0:16:44.600 that they'll just go, yeah, of course. 0:16:44.440 --> 0:16:46.400 That where it goes, right. 0:16:46.800 --> 0:16:49.200 But we it seems that we've got a grasp on 0:16:49.240 --> 0:16:52.240 this incredibly early on in a way that doesn't look 0:16:52.320 --> 0:16:53.800 like it's the ways at least most of us that 0:16:53.880 --> 0:16:56.360 taught language, right, we get quite highly told off when 0:16:56.360 --> 0:16:56.760 we try. 0:16:56.640 --> 0:16:57.680 And do expleted in fixation. 0:16:57.920 --> 0:16:59.960 Yes, So this I think would be one of those 0:17:00.160 --> 0:17:03.360 cases where you could do a sort of disnalogy by cases. Right, 0:17:03.400 --> 0:17:06.119 you present chat GPT a sentence and say insert the 0:17:06.119 --> 0:17:07.720 word fucking correctly in this sentence. 0:17:08.440 --> 0:17:10.200 And I don't think it would be very good at. 0:17:10.080 --> 0:17:10.879 It I think it would be. 0:17:10.920 --> 0:17:11.560 I think you reckon. 0:17:11.760 --> 0:17:15.280 I mean, we probably couldn't tell we could, but we shouldn't. 0:17:16.160 --> 0:17:16.400 Yeah. 0:17:16.840 --> 0:17:20.560 One of the things that I thought was like kind 0:17:20.560 --> 0:17:24.800 of interesting about how it works is that it does 0:17:24.880 --> 0:17:27.199 learn language, probably differently from the way we do, but 0:17:27.280 --> 0:17:30.040 it does it all by examples, you know, So it's 0:17:30.080 --> 0:17:32.160 looking at all these pieces of text and thinking, ah, 0:17:32.200 --> 0:17:34.960 this is okay, that's okay. And one kind of interesting 0:17:34.960 --> 0:17:38.000 thing about how humans understand language is that we're able 0:17:38.040 --> 0:17:42.800 to kind of understand meaningless but grammatical sentences. It's not 0:17:42.840 --> 0:17:46.120 clear to me that chat GPT would understand those. That's 0:17:46.160 --> 0:17:50.240 another Chopski example. So you know, Chomsky has this example 0:17:50.280 --> 0:17:57.919 that's like colorless uss sleep furiously color it's colorless green ideas, right, 0:17:58.520 --> 0:18:02.040 And that's a meaningless sentence, but it's grammatically well formed, 0:18:02.280 --> 0:18:05.119 and we can understand that it's grammatically well formed, but 0:18:05.359 --> 0:18:09.760 also that it's meaningless. Because chat Gibt kind of combines 0:18:10.520 --> 0:18:16.159 different aspects of what philosophers of language, logicians, linguistics people 0:18:17.320 --> 0:18:20.040 see as like different components of meaning. It sees these 0:18:20.080 --> 0:18:21.320 as all kind of wrapped up. 0:18:21.200 --> 0:18:23.000 In the same thing. It puts them in the same. 0:18:22.840 --> 0:18:27.199 Big model I'm not sure it could differentiate between ungrammaticality 0:18:27.240 --> 0:18:31.520 and meaninglessness. 0:18:34.080 --> 0:18:36.040 So we're doing real time science right now. 0:18:36.080 --> 0:18:38.640 I just yeah, we should check the words get. 0:18:38.520 --> 0:18:41.680 Into the following sentence in the correct way. Man, that's crazy. 0:18:42.000 --> 0:18:45.480 And I did it six times, and I would say 0:18:45.520 --> 0:18:47.400 fifty percent of the time it got it right. And 0:18:47.840 --> 0:18:50.680 it did. Man, that's fucking crazy. Man, that's crazy. Fucking Man, 0:18:50.720 --> 0:18:54.960 that's fucking crazy. Man that's crazy. Fucking My favorite is man, 0:18:55.080 --> 0:18:57.280 comma that's crazy, Comma fucking. 0:18:58.680 --> 0:18:59.320 To be fair. 0:19:01.040 --> 0:19:01.480 Reliable. 0:19:01.640 --> 0:19:03.479 You take the commas out of that last one, and 0:19:03.560 --> 0:19:04.920