1 00:00:02,440 --> 00:00:03,800 Speaker 1: Also media. 2 00:00:05,600 --> 00:00:07,760 Speaker 2: Hello and welcome to Better Offline. I'm your host ed 3 00:00:07,880 --> 00:00:09,600 Speaker 2: zitron as eva by our merchandise. 4 00:00:09,640 --> 00:00:10,760 Speaker 3: Subscribe to the newsletter. 5 00:00:10,760 --> 00:00:23,000 Speaker 4: It's all in the episode notes. 6 00:00:24,520 --> 00:00:27,840 Speaker 2: And today I'm joined by programmer Colton Bogie, who wrote 7 00:00:27,880 --> 00:00:30,360 Speaker 2: an excellent piece about a month ago called no AI 8 00:00:30,480 --> 00:00:32,519 Speaker 2: is not making engineers ten X is powerful. 9 00:00:33,479 --> 00:00:34,720 Speaker 3: Colp. Thanks for joining me. 10 00:00:34,920 --> 00:00:35,680 Speaker 1: Thank you for having me. 11 00:00:36,360 --> 00:00:38,280 Speaker 2: So tell me a little bit about your day to 12 00:00:38,360 --> 00:00:40,160 Speaker 2: day work. What do you do for a living. 13 00:00:41,240 --> 00:00:45,200 Speaker 1: Yeah, so I'm a software engineer, and I work specifically 14 00:00:45,200 --> 00:00:48,760 Speaker 1: in what's called web application development. So that's kind of 15 00:00:48,800 --> 00:00:51,760 Speaker 1: like building rich applications that work in the web browser. 16 00:00:51,840 --> 00:00:55,240 Speaker 1: So think of something like Amazon or Google Drive or 17 00:00:55,240 --> 00:00:57,080 Speaker 1: something where you know you can do a lot of 18 00:00:57,080 --> 00:01:01,840 Speaker 1: interactive features within the applihation within a browser, right, So. 19 00:01:01,840 --> 00:01:04,360 Speaker 2: Kind of like the fundament of most cloud software. 20 00:01:04,800 --> 00:01:08,160 Speaker 1: Yeah. I would say that the if not the majority 21 00:01:08,200 --> 00:01:10,959 Speaker 1: of engineers work in web application development, then probably the 22 00:01:11,000 --> 00:01:12,440 Speaker 1: plurality do right now. 23 00:01:12,480 --> 00:01:16,400 Speaker 2: And it's also a large amount of how people interact 24 00:01:16,440 --> 00:01:18,080 Speaker 2: with software is now in this way. 25 00:01:17,880 --> 00:01:22,119 Speaker 1: Though, Yeah, exactly. Yeah, and that's why that's why most 26 00:01:22,120 --> 00:01:24,480 Speaker 1: developers perhaps work in this field now. 27 00:01:25,160 --> 00:01:27,880 Speaker 2: And you'd think something like that would be perfect for 28 00:01:27,920 --> 00:01:28,640 Speaker 2: AI coding. 29 00:01:29,720 --> 00:01:29,959 Speaker 3: It is. 30 00:01:30,959 --> 00:01:35,200 Speaker 1: It is definitely the most applicable place for AI coding 31 00:01:35,280 --> 00:01:38,399 Speaker 1: of all of them. There's sort of a there's sort 32 00:01:38,440 --> 00:01:41,119 Speaker 1: of a thing where people are like, you know, oh, 33 00:01:41,240 --> 00:01:43,639 Speaker 1: you know, lms don't really work really well for my language, 34 00:01:43,640 --> 00:01:45,440 Speaker 1: and they'll be talking about something like RUST, which is 35 00:01:45,480 --> 00:01:49,120 Speaker 1: more of like a a high performance language rather than 36 00:01:49,120 --> 00:01:50,280 Speaker 1: a web language. 37 00:01:51,840 --> 00:01:54,200 Speaker 3: What is a hypophobic language in this case. 38 00:01:54,800 --> 00:01:59,320 Speaker 1: Just something that's designed to work on like weak hardware 39 00:01:59,480 --> 00:02:04,080 Speaker 1: or a like extremely high speed So like for example, 40 00:02:04,120 --> 00:02:06,240 Speaker 1: like video games aren't built in RUSS. They're usually built 41 00:02:06,240 --> 00:02:08,320 Speaker 1: in a language called C plus plus, but they're they're 42 00:02:08,320 --> 00:02:12,560 Speaker 1: built in a high performance language because you know, you're 43 00:02:12,880 --> 00:02:15,080 Speaker 1: trying to max out as much as you can do. 44 00:02:15,160 --> 00:02:18,440 Speaker 2: Ye're taking advantage of the high performance compute you have. 45 00:02:18,720 --> 00:02:21,360 Speaker 1: Yeah, whereas with a website like you know, like Amazon 46 00:02:21,440 --> 00:02:24,840 Speaker 1: or something like that, you're really not pushing the limit 47 00:02:24,919 --> 00:02:30,000 Speaker 1: in terms of interactivity, and so more modest languages are fine. 48 00:02:30,760 --> 00:02:31,200 Speaker 3: Got it? 49 00:02:31,280 --> 00:02:33,280 Speaker 2: So you laid it out really nicely in the piece, 50 00:02:33,320 --> 00:02:35,399 Speaker 2: so you can have to repeat a few things, I imagine, 51 00:02:35,480 --> 00:02:40,480 Speaker 2: but run me through why AI coding tools can't actually 52 00:02:40,520 --> 00:02:44,760 Speaker 2: make you a ten x engineer or ten times better engineer. 53 00:02:46,760 --> 00:02:52,200 Speaker 1: Yeah, so basically AI is really good at AI is 54 00:02:52,240 --> 00:02:56,400 Speaker 1: almost shockingly good at generating code, and it's almost shockingly 55 00:02:56,400 --> 00:03:03,000 Speaker 1: good at generating code that runs, and it can answer 56 00:03:03,040 --> 00:03:05,520 Speaker 1: a lot of questions that are that are like difficult 57 00:03:06,360 --> 00:03:09,959 Speaker 1: or at least, you know, annoying is probably the better word. 58 00:03:09,960 --> 00:03:11,680 Speaker 1: It's really good at dealing with things that are annoying. 59 00:03:12,040 --> 00:03:14,680 Speaker 1: It's really good at, you know, so encoding. We have 60 00:03:14,720 --> 00:03:18,079 Speaker 1: this concept called boilerplate, and it's it's just code that 61 00:03:18,120 --> 00:03:20,560 Speaker 1: you have to kind of rewrite a lot. And as 62 00:03:20,600 --> 00:03:23,240 Speaker 1: an engineer, ideally you don't have to write a lot 63 00:03:23,240 --> 00:03:26,240 Speaker 1: of boilerplate. Ideal you automate things and distract things, so 64 00:03:26,320 --> 00:03:28,920 Speaker 1: you you don't need to do that very often. But 65 00:03:29,080 --> 00:03:31,640 Speaker 1: it's sort of like a requirement of the job. And 66 00:03:31,720 --> 00:03:34,359 Speaker 1: it's really good at writing that because you know, it's 67 00:03:34,480 --> 00:03:38,520 Speaker 1: just it's kind of like low intent, high volume code, 68 00:03:39,720 --> 00:03:43,280 Speaker 1: quantity over quality type thing. So it's really good. It's 69 00:03:43,320 --> 00:03:47,680 Speaker 1: stuff like that. The problem is that generating code really 70 00:03:47,760 --> 00:03:52,120 Speaker 1: isn't the hard part of being a software engineer. It's 71 00:03:52,160 --> 00:03:54,840 Speaker 1: it's one of the things that really matters, and it's like, 72 00:03:54,880 --> 00:03:57,480 Speaker 1: obviously if you go to college to learn how to code, 73 00:03:57,480 --> 00:03:58,960 Speaker 1: what you're going to spend most of the time doing 74 00:03:59,040 --> 00:04:02,480 Speaker 1: is is typing code, but it's it's really just like 75 00:04:02,600 --> 00:04:06,000 Speaker 1: one thing that you're doing, Like in reality, you're doing 76 00:04:06,000 --> 00:04:07,880 Speaker 1: a lot more thinking about like, Okay, how does this 77 00:04:08,120 --> 00:04:10,520 Speaker 1: work with the systems already have? How do I avoid 78 00:04:11,160 --> 00:04:14,400 Speaker 1: creating what's called tech debt? So tech debt is basically 79 00:04:14,440 --> 00:04:16,440 Speaker 1: like an easy way of thinking about it is when 80 00:04:16,480 --> 00:04:19,360 Speaker 1: you write code that makes writing future code harder. So 81 00:04:19,480 --> 00:04:22,720 Speaker 1: an example would be, you know, you want to write 82 00:04:22,920 --> 00:04:25,799 Speaker 1: like a piece of logic that, let's say computes sales tax. 83 00:04:25,839 --> 00:04:27,640 Speaker 1: If you're, you know, making an online story, you'd want 84 00:04:27,640 --> 00:04:30,080 Speaker 1: to compute sales tax based on where they are. You 85 00:04:30,120 --> 00:04:31,719 Speaker 1: only want to write that once. You don't want to 86 00:04:31,720 --> 00:04:34,839 Speaker 1: have two different places that you know handle sales tax, 87 00:04:34,880 --> 00:04:38,520 Speaker 1: because then, if you know, Illinois changes their sales tax, right, 88 00:04:38,520 --> 00:04:40,360 Speaker 1: you have to change two places instead of one. 89 00:04:41,000 --> 00:04:44,039 Speaker 2: It doesn't doesn't The very nature of code also mean 90 00:04:44,080 --> 00:04:46,400 Speaker 2: that it naturally creates tech debt. 91 00:04:46,920 --> 00:04:49,920 Speaker 1: You know, you can try to mitigate as much as 92 00:04:49,920 --> 00:04:52,120 Speaker 1: you can, but there's not even an idea like that 93 00:04:52,680 --> 00:04:57,320 Speaker 1: all code is tech debt. And and what LLLMS so 94 00:04:57,640 --> 00:04:59,880 Speaker 1: kind of what I mentioned before. You know, lms are 95 00:05:00,080 --> 00:05:02,240 Speaker 1: get at producing boilerplate. But if you get to the 96 00:05:02,279 --> 00:05:04,880 Speaker 1: point where boilerplate is really easy to produce and you're 97 00:05:04,920 --> 00:05:08,280 Speaker 1: not constantly thinking like, Okay, how do I avoid doing 98 00:05:08,320 --> 00:05:12,159 Speaker 1: this in the future, you start, yeah, exactly, you start 99 00:05:12,200 --> 00:05:15,159 Speaker 1: generating more and more code. And just like generating code 100 00:05:15,240 --> 00:05:17,480 Speaker 1: is an it's kind of like writing in terms of, 101 00:05:17,520 --> 00:05:20,119 Speaker 1: like you know, writing a book, like a good writer 102 00:05:20,800 --> 00:05:22,520 Speaker 1: writes fewer words rather than more. 103 00:05:22,760 --> 00:05:24,400 Speaker 3: Yeah. Yeah, brevity is the soul of wit. 104 00:05:24,800 --> 00:05:28,159 Speaker 2: Now. It's because what I've been realizing is that there 105 00:05:28,200 --> 00:05:31,400 Speaker 2: is this delta between people that actually write software and 106 00:05:31,440 --> 00:05:34,280 Speaker 2: people that are excited about AI coding. Because I had 107 00:05:34,360 --> 00:05:36,919 Speaker 2: Carl Brown from the Internet of Bugs on and it 108 00:05:37,000 --> 00:05:39,320 Speaker 2: was this thing of yeah, being the software engineer is 109 00:05:39,360 --> 00:05:42,360 Speaker 2: not just writing one hundred lines of code and then 110 00:05:42,720 --> 00:05:45,000 Speaker 2: giving the thumbs up to your boss. You have to 111 00:05:45,080 --> 00:05:49,359 Speaker 2: interact with various different parts of the organization. Early on 112 00:05:49,400 --> 00:05:50,920 Speaker 2: in the piece as well, you had this bit called 113 00:05:50,920 --> 00:05:53,240 Speaker 2: the math no link to this obviously in the episode notes, 114 00:05:53,240 --> 00:05:56,400 Speaker 2: where it's any software engineer has worked on actual code 115 00:05:56,400 --> 00:05:59,560 Speaker 2: in the natural company knows that you have these steps 116 00:06:00,120 --> 00:06:02,400 Speaker 2: where it isn't just like icode and then I give 117 00:06:02,440 --> 00:06:03,960 Speaker 2: the thumbs up and I'm dumb for the day. You 118 00:06:04,000 --> 00:06:06,400 Speaker 2: have to go through a reviewer, you have to wait 119 00:06:06,440 --> 00:06:08,560 Speaker 2: for them to get back to you. You have to 120 00:06:08,600 --> 00:06:10,919 Speaker 2: contact switching as well. We have to change different windows 121 00:06:10,920 --> 00:06:13,520 Speaker 2: and do different things. There's a shit ton of just 122 00:06:13,600 --> 00:06:17,599 Speaker 2: intermediary work that is nothing to do with actually writing code, right. 123 00:06:18,720 --> 00:06:22,920 Speaker 1: Yeah, absolutely, I mean it's it really is more of 124 00:06:23,240 --> 00:06:25,760 Speaker 1: you know, there's parts of coding that is more science 125 00:06:25,760 --> 00:06:28,040 Speaker 1: than there's parts of coding that are more art and 126 00:06:28,160 --> 00:06:30,760 Speaker 1: more you know, in college, is really common if you're 127 00:06:30,760 --> 00:06:34,159 Speaker 1: getting computer science deority to take communications classes because you 128 00:06:34,240 --> 00:06:35,840 Speaker 1: just have to interact with a lot of people. So 129 00:06:36,120 --> 00:06:40,320 Speaker 1: the standard way that a thing gets done in a 130 00:06:40,360 --> 00:06:43,640 Speaker 1: software project is you have somebody called a product manager, 131 00:06:43,720 --> 00:06:47,200 Speaker 1: and that person's that person's job is to think about 132 00:06:47,240 --> 00:06:50,440 Speaker 1: what the product is as it exists now and what 133 00:06:50,480 --> 00:06:52,480 Speaker 1: should be in the future, and like what features we 134 00:06:52,480 --> 00:06:55,240 Speaker 1: should build, Like basically more than anything, it's you know, 135 00:06:55,279 --> 00:06:57,080 Speaker 1: what what should we build next? And then you have 136 00:06:57,120 --> 00:06:59,160 Speaker 1: designers who are going to decide how that thing should look. 137 00:06:59,520 --> 00:07:03,480 Speaker 1: And engineers have to be this meeting influence because they're 138 00:07:03,520 --> 00:07:06,640 Speaker 1: the only ones in the process who actually know what 139 00:07:06,680 --> 00:07:08,880 Speaker 1: it's like, how hard it is going to be to 140 00:07:08,880 --> 00:07:12,040 Speaker 1: build something. So a frequent interaction is like, you know, 141 00:07:12,080 --> 00:07:13,840 Speaker 1: product maators are like, oh, we should add this, and 142 00:07:13,880 --> 00:07:16,440 Speaker 1: an engineer has to kind of step in and be like, no, 143 00:07:16,560 --> 00:07:20,000 Speaker 1: that's borderline impossible or you know that would take six months, 144 00:07:21,000 --> 00:07:23,000 Speaker 1: right and you want coding. 145 00:07:23,120 --> 00:07:25,960 Speaker 2: Lllms don't fix that problem. They don't do they actually 146 00:07:25,960 --> 00:07:27,960 Speaker 2: make it easier to develop products. 147 00:07:28,200 --> 00:07:33,400 Speaker 1: I think there are uses for LMS in coding, and. 148 00:07:33,440 --> 00:07:38,240 Speaker 2: I'm not even denying that. I'm just like, how, like, 149 00:07:38,280 --> 00:07:41,280 Speaker 2: what do what do those uses end up actually creating? 150 00:07:42,280 --> 00:07:46,200 Speaker 1: Basically, you know, what I found is that I don't 151 00:07:46,200 --> 00:07:50,640 Speaker 1: really like using lms for most product work, even though 152 00:07:50,680 --> 00:07:52,360 Speaker 1: they're good at you know, say like oh add a 153 00:07:52,360 --> 00:07:54,760 Speaker 1: button here that does this. They can be good at that, 154 00:07:54,880 --> 00:07:58,200 Speaker 1: especially if you have a codebase that that works really 155 00:07:58,280 --> 00:08:02,920 Speaker 1: well with lllms. So some programming languages, some tools, some 156 00:08:03,200 --> 00:08:05,640 Speaker 1: we call them libraries and coding you know, they're basically 157 00:08:05,680 --> 00:08:09,000 Speaker 1: like shared software. Some of those things work really well 158 00:08:09,040 --> 00:08:12,040 Speaker 1: with lms because the lms are just trained on the Internet. 159 00:08:13,120 --> 00:08:16,280 Speaker 1: So sites like stack overflow and sites like leak code 160 00:08:16,320 --> 00:08:18,680 Speaker 1: and stuff like that, and so if they have a 161 00:08:18,760 --> 00:08:22,440 Speaker 1: large body of this code to work within their training data, 162 00:08:22,520 --> 00:08:25,440 Speaker 1: like these tools and these languages, they tend to be 163 00:08:25,440 --> 00:08:28,680 Speaker 1: better at writing them. And so you know, I work 164 00:08:28,920 --> 00:08:32,400 Speaker 1: primarily in JavaScript, as do most web application engineers, and 165 00:08:32,880 --> 00:08:35,640 Speaker 1: lms are quite good at JavaScript, but I still don't 166 00:08:35,720 --> 00:08:38,000 Speaker 1: like to use that much because it tends to just 167 00:08:38,120 --> 00:08:41,760 Speaker 1: not understand context very well. 168 00:08:43,040 --> 00:08:44,800 Speaker 3: And mean impracticality. 169 00:08:45,520 --> 00:08:47,800 Speaker 1: So it's kind of just like you know, understanding the 170 00:08:48,280 --> 00:08:50,640 Speaker 1: existing resources that you've already built in your code base. 171 00:08:50,720 --> 00:08:52,840 Speaker 1: So this is like the avoiding writing the same like 172 00:08:52,920 --> 00:08:56,640 Speaker 1: sales tax thing twice. They tend to default to rewriting 173 00:08:56,720 --> 00:09:03,640 Speaker 1: things and they tend to struggle to reuse the same 174 00:09:03,960 --> 00:09:07,319 Speaker 1: like style. So you know, an important thing in coding 175 00:09:07,559 --> 00:09:11,640 Speaker 1: is maintaining consistent like styles of your code. So like 176 00:09:11,679 --> 00:09:16,600 Speaker 1: there's there's a whole theory and practice of actual like 177 00:09:16,720 --> 00:09:20,960 Speaker 1: how your code should look like as like how the texture, 178 00:09:21,040 --> 00:09:23,560 Speaker 1: like how you should avoid, what things you should avoid, 179 00:09:23,600 --> 00:09:27,160 Speaker 1: because there's there's bad patterns, there's good patterns, and you know, 180 00:09:27,160 --> 00:09:31,040 Speaker 1: there's everything in between. I think it's very similar to 181 00:09:31,160 --> 00:09:35,880 Speaker 1: like you know, when image generation AI kind of came 182 00:09:35,920 --> 00:09:38,040 Speaker 1: onto the scene, you had a lot of people being like, 183 00:09:38,040 --> 00:09:40,240 Speaker 1: all right, well, graphic designers are done. You know, they're 184 00:09:40,240 --> 00:09:42,000 Speaker 1: out of a job. They don't need to do anything 185 00:09:42,200 --> 00:09:44,040 Speaker 1: because I can generate a logo now and like it 186 00:09:44,040 --> 00:09:46,079 Speaker 1: looks pretty good. Like I can generate a logo for 187 00:09:46,160 --> 00:09:48,320 Speaker 1: my nail slam or whatever, and look at it looks great. 188 00:09:48,760 --> 00:09:51,200 Speaker 1: But as soon as you try to generate a second 189 00:09:51,640 --> 00:09:54,120 Speaker 1: logo that looks like the first one and you know, 190 00:09:54,160 --> 00:09:56,840 Speaker 1: maybe has a slight modification or or something like that, 191 00:09:57,240 --> 00:09:59,760 Speaker 1: it's really it's really bad at it. It's really bad 192 00:09:59,760 --> 00:10:01,679 Speaker 1: at you using the context and you're like, okay, now 193 00:10:01,679 --> 00:10:03,640 Speaker 1: I need a graphic designer. And as soon as you 194 00:10:03,679 --> 00:10:09,760 Speaker 1: need you know one anyway, yeah, exactly, and so and 195 00:10:09,960 --> 00:10:12,760 Speaker 1: and you know that's it's very similar to engineers, you know, 196 00:10:12,760 --> 00:10:15,280 Speaker 1: they a lot of what you're doing is sort of 197 00:10:15,320 --> 00:10:20,480 Speaker 1: working around context and avoiding redoing things and avoiding moving 198 00:10:20,559 --> 00:10:22,719 Speaker 1: away from your existing styles. 199 00:10:23,160 --> 00:10:26,400 Speaker 2: And what is the critical nature of styles? 200 00:10:26,480 --> 00:10:26,760 Speaker 3: Is that? 201 00:10:26,800 --> 00:10:30,800 Speaker 2: Because is that so the people looking at your staff 202 00:10:30,920 --> 00:10:33,280 Speaker 2: in the future can say, Okay, this is what they 203 00:10:33,320 --> 00:10:36,400 Speaker 2: were going for. This is so that everything doesn't break. 204 00:10:36,800 --> 00:10:40,840 Speaker 1: Yes, exactly. It's about consistency. It's about so there are 205 00:10:40,880 --> 00:10:43,520 Speaker 1: certain you know, for example, JavaScript, which is the main 206 00:10:43,600 --> 00:10:49,920 Speaker 1: language I work in, is almost infamous for basically allowing 207 00:10:49,960 --> 00:10:53,600 Speaker 1: you as the developer to do all sorts of buck 208 00:10:53,679 --> 00:10:55,920 Speaker 1: wild stuff that you should like never do coding wise, 209 00:10:55,960 --> 00:10:59,480 Speaker 1: like a lot of old patterns, a lot of recycled stuff, 210 00:10:59,559 --> 00:11:01,760 Speaker 1: a lot of it just it just lets you. It's 211 00:11:01,880 --> 00:11:05,920 Speaker 1: it's an extremely varied language in the things it supports, 212 00:11:05,960 --> 00:11:07,480 Speaker 1: and so what you want to do when you're writing 213 00:11:07,480 --> 00:11:10,880 Speaker 1: in JavaScript is only use the good parts. You want 214 00:11:10,920 --> 00:11:13,760 Speaker 1: to strategically avoid doing a bunch of bad things. And 215 00:11:13,840 --> 00:11:16,760 Speaker 1: some of those bad things, there's tools that will automatically 216 00:11:16,800 --> 00:11:29,280 Speaker 1: detect if you're doing them. 217 00:11:29,440 --> 00:11:31,360 Speaker 3: What are these bad things? Is it just the job? 218 00:11:31,679 --> 00:11:34,280 Speaker 3: What's so special about Java that it allows you to 219 00:11:34,640 --> 00:11:34,960 Speaker 3: do that? 220 00:11:35,640 --> 00:11:40,880 Speaker 1: Yeah, JavaScript, which is technically different from Java, Yeah, no problem. Basically, 221 00:11:40,960 --> 00:11:43,320 Speaker 1: the reason JavaScript has all these bad parts is because 222 00:11:43,360 --> 00:11:47,240 Speaker 1: it was created in ten days as a toy language 223 00:11:47,400 --> 00:11:50,640 Speaker 1: at Mosaic, which was the precursor to Firefox, you know, 224 00:11:50,679 --> 00:11:55,200 Speaker 1: the first browser that really green traction and sat the Internet. 225 00:11:55,360 --> 00:11:57,440 Speaker 2: I'm old enough to have used Mosaic somehow. 226 00:11:57,840 --> 00:12:01,240 Speaker 1: Yeah. So JavaScript was created in like ten days almost 227 00:12:01,400 --> 00:12:03,680 Speaker 1: like just as like a thing to test, like just 228 00:12:03,679 --> 00:12:05,760 Speaker 1: something to try like, okay, what if we put a 229 00:12:05,800 --> 00:12:09,160 Speaker 1: coding language into the into the browser that people could 230 00:12:09,200 --> 00:12:14,640 Speaker 1: just like ship with their websites. And in response to that, 231 00:12:14,840 --> 00:12:18,559 Speaker 1: other company said, okay, well we need to support JavaScript 232 00:12:18,559 --> 00:12:20,760 Speaker 1: as well, and so when Internet Explorer came out, they 233 00:12:20,800 --> 00:12:23,439 Speaker 1: shipped their own version of JavaScript, which was slightly different. 234 00:12:24,000 --> 00:12:26,640 Speaker 1: And so you have this engineer that was kind of 235 00:12:26,640 --> 00:12:30,160 Speaker 1: built on like these like very hacked to get their principles, 236 00:12:30,440 --> 00:12:33,920 Speaker 1: that was then rebuilt with slightly different principles, and then 237 00:12:34,200 --> 00:12:36,920 Speaker 1: what you have now is thirty years later, sort of 238 00:12:37,480 --> 00:12:41,280 Speaker 1: the conglomeration of all those things into one language, and 239 00:12:41,880 --> 00:12:45,480 Speaker 1: it's improved dramatically in that time. But the nature of 240 00:12:45,520 --> 00:12:48,240 Speaker 1: coding languages is there's a lot of backwards compatibility. Like 241 00:12:48,240 --> 00:12:51,000 Speaker 1: a video game where you know, you you know, you 242 00:12:51,040 --> 00:12:53,080 Speaker 1: want to be able to build a computer that can 243 00:12:53,160 --> 00:12:56,040 Speaker 1: run a game from nineteen ninety one, so you want 244 00:12:56,040 --> 00:12:58,079 Speaker 1: the browser to be able to still run code largely 245 00:12:58,120 --> 00:13:00,880 Speaker 1: that was written in nineteen ninety five or whatever. So 246 00:13:01,120 --> 00:13:03,560 Speaker 1: a lot of that stuff still exists, and what they've 247 00:13:03,559 --> 00:13:06,680 Speaker 1: done is introduce new patterns that can be better, and 248 00:13:06,720 --> 00:13:09,440 Speaker 1: so there's a sort of an equal amount of like 249 00:13:09,520 --> 00:13:13,800 Speaker 1: avoiding that and then also just consistency. So there are 250 00:13:14,120 --> 00:13:17,640 Speaker 1: times where there's two good ways to write ways to 251 00:13:17,720 --> 00:13:20,120 Speaker 1: do something, but you always want to do it only 252 00:13:20,160 --> 00:13:22,600 Speaker 1: in one way so that anytime someone reads it, they 253 00:13:22,640 --> 00:13:24,360 Speaker 1: see that pattern and they say, okay, I know exactly 254 00:13:24,440 --> 00:13:25,240 Speaker 1: what this does. 255 00:13:25,600 --> 00:13:29,920 Speaker 2: Right, And this fundamentally feels like something large of language 256 00:13:29,920 --> 00:13:32,960 Speaker 2: world is bad at because they don't know anything and 257 00:13:32,960 --> 00:13:35,480 Speaker 2: they don't create anything unique either. 258 00:13:35,520 --> 00:13:37,720 Speaker 3: They just repeat yeah exactly. 259 00:13:37,760 --> 00:13:42,400 Speaker 1: I mean, they are statistical inference models, and so they 260 00:13:42,480 --> 00:13:46,200 Speaker 1: are very good at generating what they think should come 261 00:13:46,240 --> 00:13:51,320 Speaker 1: next based on probabilities. And you can when you're training 262 00:13:51,480 --> 00:13:53,240 Speaker 1: a large language model, you can try to push it 263 00:13:53,240 --> 00:13:55,000 Speaker 1: in one way, push it in another way to say 264 00:13:55,000 --> 00:13:56,920 Speaker 1: like no, don't do that, do that. But like trying 265 00:13:56,960 --> 00:13:59,480 Speaker 1: to do that on the broad spectrum of all things 266 00:13:59,559 --> 00:14:02,640 Speaker 1: code is impossible, and so you're going to get things 267 00:14:02,679 --> 00:14:05,320 Speaker 1: that default to the way that they were done on 268 00:14:05,360 --> 00:14:08,000 Speaker 1: the Internet, and so much like so, like like I said, 269 00:14:08,080 --> 00:14:12,560 Speaker 1: JavaScript it is thirty years old, it's extremely it's gone 270 00:14:12,600 --> 00:14:15,160 Speaker 1: through a lot of turk times in terms of patterns 271 00:14:15,480 --> 00:14:19,280 Speaker 1: and so on the Internet, there are just fast swaths 272 00:14:19,320 --> 00:14:21,720 Speaker 1: of terrible JavaScript and you know. 273 00:14:22,000 --> 00:14:24,320 Speaker 2: So it's just these models are trained on bad code 274 00:14:24,320 --> 00:14:25,480 Speaker 2: as well as good code. 275 00:14:25,840 --> 00:14:29,560 Speaker 1: Yeah, exactly, and and that you know, I'm sure they're 276 00:14:29,560 --> 00:14:33,160 Speaker 1: trying very hard when they're training the models to filter 277 00:14:33,240 --> 00:14:34,600 Speaker 1: some of this stuff out, but like trying to do 278 00:14:34,640 --> 00:14:39,080 Speaker 1: a broad filter on hundreds of thousands, millions even of 279 00:14:39,200 --> 00:14:43,320 Speaker 1: what does say again, like decades, Like I mean, JavaScript 280 00:14:43,360 --> 00:14:45,680 Speaker 1: has been around for thirty years. I just have most 281 00:14:45,960 --> 00:14:48,320 Speaker 1: programming languages that are in use right now. 282 00:14:48,440 --> 00:14:52,920 Speaker 2: Right, but this particular one seems particularly chaotic in how 283 00:14:53,000 --> 00:14:53,600 Speaker 2: it's sprawled. 284 00:14:54,160 --> 00:14:57,600 Speaker 1: Yes, JavaScript is a particularly thirty language. Yeah. 285 00:14:57,720 --> 00:15:00,600 Speaker 2: Why So what you're describing, I'm not trying to put 286 00:15:00,600 --> 00:15:03,040 Speaker 2: words in your mouth, is that this stuff doesn't do 287 00:15:03,160 --> 00:15:05,280 Speaker 2: the stuff that everyone's saying it does. Like it's not 288 00:15:05,400 --> 00:15:09,720 Speaker 2: replacing engineers. It doesn't even seem like it could replace engineers. 289 00:15:10,120 --> 00:15:12,640 Speaker 2: Like it's just not it doesn't do In fact, maybe 290 00:15:12,680 --> 00:15:15,520 Speaker 2: it's fair to say that. So it doesn't do software engineering. 291 00:15:16,400 --> 00:15:19,240 Speaker 1: That's almost the perfect way to put it. It does coding, 292 00:15:19,480 --> 00:15:22,720 Speaker 1: but it doesn't do software engineering, and software engineering is 293 00:15:22,920 --> 00:15:25,680 Speaker 1: kind of this broader practice of like everything that comes 294 00:15:25,720 --> 00:15:30,800 Speaker 1: together around coding. So you know, some people really integrate 295 00:15:31,040 --> 00:15:35,440 Speaker 1: lms deeply into their everyday work and they do similar 296 00:15:35,480 --> 00:15:38,000 Speaker 1: work to what I do. So you know, there are 297 00:15:38,040 --> 00:15:42,760 Speaker 1: people who who are primarily having like lms write their code, 298 00:15:42,800 --> 00:15:48,880 Speaker 1: and they can be good engineers, but they're intervening pretty 299 00:15:48,880 --> 00:15:52,000 Speaker 1: constantly from what I understand, and you know, they're having 300 00:15:52,040 --> 00:15:54,400 Speaker 1: to sort of redirect it, make sure it stays on 301 00:15:54,440 --> 00:15:58,240 Speaker 1: patterns and all that stuff, and there's just kind of 302 00:15:58,280 --> 00:15:59,800 Speaker 1: you know, this is the reason that I don't really 303 00:16:00,360 --> 00:16:03,880 Speaker 1: elms that much, is there's just a constant tension between 304 00:16:04,400 --> 00:16:09,160 Speaker 1: the the lack of context they have and you know, 305 00:16:09,200 --> 00:16:11,360 Speaker 1: what you want them to do and constantly reviewing the 306 00:16:11,400 --> 00:16:13,840 Speaker 1: code and making sure it's up to standards when I 307 00:16:13,880 --> 00:16:15,560 Speaker 1: know I could just write it and you know it'll 308 00:16:15,560 --> 00:16:17,720 Speaker 1: take up out as much time then as it would 309 00:16:17,720 --> 00:16:21,400 Speaker 1: take prompting and re prompting. It's just I get better 310 00:16:21,440 --> 00:16:23,520 Speaker 1: code that way, and that's that's what I care about most. 311 00:16:23,640 --> 00:16:26,160 Speaker 2: So do you think it's kind of a mirage al 312 00:16:26,200 --> 00:16:28,040 Speaker 2: most the productivity benefits? 313 00:16:28,760 --> 00:16:32,320 Speaker 1: I think it would vary a lot. I think yes, broadly, 314 00:16:32,720 --> 00:16:35,240 Speaker 1: I think that there are productivity benefits, but like these 315 00:16:35,320 --> 00:16:38,600 Speaker 1: like huge outsides, like oh my gosh, I'm galaxy brained 316 00:16:38,640 --> 00:16:41,800 Speaker 1: right now, I'm doing so much. I think that that 317 00:16:42,320 --> 00:16:46,840 Speaker 1: is largely a mirage based on like extrapolation of like 318 00:16:47,000 --> 00:16:50,440 Speaker 1: small wins. I talk about this a little bit in 319 00:16:50,440 --> 00:16:53,440 Speaker 1: my article that you know, there are times. One thing 320 00:16:53,480 --> 00:16:56,560 Speaker 1: that I use LMS for a lot and not a 321 00:16:56,600 --> 00:16:59,920 Speaker 1: lot sometimes, but that they are really good at is 322 00:17:00,160 --> 00:17:02,640 Speaker 1: you know, sometimes you're writing code and you're like, I 323 00:17:02,680 --> 00:17:04,520 Speaker 1: need to write a thing that I will only run 324 00:17:04,560 --> 00:17:06,800 Speaker 1: once and I will throw in the trash, or I 325 00:17:06,800 --> 00:17:09,960 Speaker 1: need to write a thing and it uses this tool 326 00:17:10,200 --> 00:17:13,199 Speaker 1: that like I don't have the time to learn, but 327 00:17:13,280 --> 00:17:15,280 Speaker 1: like I'd really like to have this tool, Like I'd 328 00:17:15,320 --> 00:17:19,159 Speaker 1: really like to just like use this once and so 329 00:17:19,320 --> 00:17:21,320 Speaker 1: you can you can vibe code something and not really 330 00:17:21,440 --> 00:17:23,400 Speaker 1: understand what the code does. You can run it once 331 00:17:23,520 --> 00:17:26,000 Speaker 1: and not you know, validate the output and make sure 332 00:17:26,040 --> 00:17:29,680 Speaker 1: it works fine, and you know I've saved in that time. 333 00:17:29,800 --> 00:17:32,399 Speaker 1: You know, it might have taken me five hours to 334 00:17:32,480 --> 00:17:34,679 Speaker 1: like learn how to use this tool properly and like 335 00:17:34,760 --> 00:17:37,160 Speaker 1: learn how to use it with good standards. It could 336 00:17:37,320 --> 00:17:39,560 Speaker 1: even have taken me more, and instead I spent twenty minutes. 337 00:17:40,240 --> 00:17:42,480 Speaker 2: You know, wouldn't that be dangerous because you don't know 338 00:17:42,520 --> 00:17:45,560 Speaker 2: how it works exactly. 339 00:17:45,400 --> 00:17:47,160 Speaker 1: Exactly why I like to only use it for these 340 00:17:47,160 --> 00:17:51,480 Speaker 1: like one time like low. You know. So for example, 341 00:17:51,600 --> 00:17:54,240 Speaker 1: I wrote some code the other week and I was 342 00:17:54,280 --> 00:17:58,320 Speaker 1: basically I basically refactored some existing code. So I adjusted 343 00:17:58,320 --> 00:18:00,720 Speaker 1: how it worked a little bit, and I realized there 344 00:18:00,760 --> 00:18:04,800 Speaker 1: was a way that I could break some existing code 345 00:18:05,080 --> 00:18:07,240 Speaker 1: with it. But I didn't have an easy way of 346 00:18:07,320 --> 00:18:10,520 Speaker 1: checking across the entire code based. So we have tests 347 00:18:10,560 --> 00:18:12,919 Speaker 1: in our code base that would you know, catch issues, 348 00:18:12,920 --> 00:18:15,359 Speaker 1: but there's always some code that isn't quite up to 349 00:18:15,640 --> 00:18:18,480 Speaker 1: up to par with tests. And so I had this 350 00:18:18,560 --> 00:18:21,360 Speaker 1: idea that you know, a simple a really simple language 351 00:18:21,359 --> 00:18:24,320 Speaker 1: parser could go through and make sure that this code 352 00:18:24,359 --> 00:18:26,840 Speaker 1: was right, you know, something that is only like thirty 353 00:18:26,880 --> 00:18:30,639 Speaker 1: lines of code. But language parsing is really complicated, and 354 00:18:30,800 --> 00:18:33,560 Speaker 1: the tools to do it are you know there, there 355 00:18:33,600 --> 00:18:36,320 Speaker 1: are a lot of them, and they're very well supported. 356 00:18:36,359 --> 00:18:40,320 Speaker 1: And because language parsing is a huge deal, but it 357 00:18:40,359 --> 00:18:41,960 Speaker 1: would it would take me a lot of time to 358 00:18:42,040 --> 00:18:43,680 Speaker 1: learn that. So I just I just vibe coded a 359 00:18:43,720 --> 00:18:46,680 Speaker 1: little tool. I said, hey, find me every case where 360 00:18:46,680 --> 00:18:48,680 Speaker 1: I use this function, and I call it like this 361 00:18:49,680 --> 00:18:52,480 Speaker 1: in this entire code base, or actually said, I said, 362 00:18:52,480 --> 00:18:55,199 Speaker 1: write a script that will find me that and I 363 00:18:55,240 --> 00:18:56,800 Speaker 1: looked at the code, I was like, yeah, that looks right, 364 00:18:56,840 --> 00:18:59,720 Speaker 1: that looks right, looks right. I ran it, it said 365 00:18:59,720 --> 00:19:01,639 Speaker 1: there was there's no issues in my code base. So 366 00:19:01,680 --> 00:19:04,160 Speaker 1: I intentionally created an issue just to make sure it worked. 367 00:19:04,400 --> 00:19:07,000 Speaker 1: And it worked, it showed it, and so then I 368 00:19:07,040 --> 00:19:08,640 Speaker 1: got rid of the intentional issue. I was like, Okay, 369 00:19:08,680 --> 00:19:10,440 Speaker 1: this is probably good, and I pushed my code and 370 00:19:10,480 --> 00:19:11,320 Speaker 1: it turned out to be good. 371 00:19:11,480 --> 00:19:14,919 Speaker 3: But that's you know, that also seems low stakes. 372 00:19:15,080 --> 00:19:18,000 Speaker 1: Exactly, and it's it's what I would say, wouldn't scale. 373 00:19:18,040 --> 00:19:21,879 Speaker 1: So if I really needed to start doing like language 374 00:19:21,880 --> 00:19:24,720 Speaker 1: person constantly, like I was doing it daily at my job, 375 00:19:25,080 --> 00:19:28,520 Speaker 1: I simply would have to learn. Like you said, you know, 376 00:19:28,680 --> 00:19:30,520 Speaker 1: how do you know that there are any issues? I 377 00:19:30,520 --> 00:19:33,879 Speaker 1: would have to learn the tools, right, So the time 378 00:19:33,920 --> 00:19:39,440 Speaker 1: that I saved was by avoiding learning for this one thing. 379 00:19:39,560 --> 00:19:42,360 Speaker 1: But eventually, like if you're going to make something your 380 00:19:42,359 --> 00:19:44,119 Speaker 1: full time job, you have to learn it because you 381 00:19:44,160 --> 00:19:46,280 Speaker 1: can't fully trust the output. 382 00:19:46,080 --> 00:19:50,120 Speaker 3: Because it isn't your job, like you were just kind 383 00:19:50,160 --> 00:19:51,240 Speaker 3: of mimicking. 384 00:19:51,720 --> 00:19:55,520 Speaker 1: Yeah, and you know you have to. You know, the 385 00:19:55,760 --> 00:20:01,480 Speaker 1: llms make mistakes in occasionally extremely catastrophic ways. There's a 386 00:20:01,640 --> 00:20:04,160 Speaker 1: thing called slop squatting. Have you heard of slap squatting? 387 00:20:04,440 --> 00:20:06,399 Speaker 3: No, but please tell me. I love this term so 388 00:20:06,520 --> 00:20:07,280 Speaker 3: much already. 389 00:20:07,640 --> 00:20:11,040 Speaker 1: Yeah. So basically you might you might have heard like 390 00:20:11,119 --> 00:20:12,160 Speaker 1: domain squatting. 391 00:20:12,240 --> 00:20:13,960 Speaker 2: So this is where like I think I know what 392 00:20:14,000 --> 00:20:15,840 Speaker 2: this is and I'm very excited to hear more. 393 00:20:16,359 --> 00:20:18,200 Speaker 1: Yeah. So, like you know, this was a thing where 394 00:20:18,240 --> 00:20:21,920 Speaker 1: like in nineteen, you know, ninety one, you're like, Okay, 395 00:20:21,960 --> 00:20:23,840 Speaker 1: I think the Internet's gonna be big, so I'm gonna 396 00:20:23,840 --> 00:20:26,639 Speaker 1: grab Nike dot com. Right, I'm just gonna hold it. 397 00:20:26,680 --> 00:20:30,679 Speaker 1: So that's squatting a domain. So slap squatting is basically 398 00:20:30,800 --> 00:20:36,919 Speaker 1: where you the the lms. You know, they're statistical inference machines. 399 00:20:36,920 --> 00:20:41,200 Speaker 1: They don't they don't actually understand uh, what they're doing. 400 00:20:41,280 --> 00:20:45,520 Speaker 1: And so you know, sometimes they will import software. 401 00:20:45,840 --> 00:20:49,960 Speaker 2: They will oh is this when it looks on GitHub 402 00:20:50,000 --> 00:20:52,160 Speaker 2: for something that doesn't exist. 403 00:20:52,480 --> 00:20:56,399 Speaker 1: Yes, and and it it It will just add code 404 00:20:56,400 --> 00:20:59,520 Speaker 1: to your project that's like import this thing and it 405 00:20:59,560 --> 00:21:02,159 Speaker 1: will stall it and it'll say like, okay, this is 406 00:21:02,200 --> 00:21:04,160 Speaker 1: the library that you want to use, and it's it's 407 00:21:04,200 --> 00:21:07,000 Speaker 1: not right. It's it's either misspelled or it's you know, 408 00:21:07,560 --> 00:21:09,880 Speaker 1: for example, you would be looking for a library called 409 00:21:10,359 --> 00:21:13,400 Speaker 1: uh left dash pad, and it would import something called 410 00:21:13,520 --> 00:21:18,000 Speaker 1: left pad with no dash. And so what people realized 411 00:21:18,119 --> 00:21:21,480 Speaker 1: is that they can, you know, when the the lms, 412 00:21:21,480 --> 00:21:25,359 Speaker 1: there's statistical machines, and so they frequently they do the 413 00:21:25,400 --> 00:21:27,640 Speaker 1: same thing a lot, they make the same mistakes a lot. 414 00:21:27,680 --> 00:21:31,800 Speaker 1: So you could grab that library left pad with no 415 00:21:31,960 --> 00:21:36,679 Speaker 1: dago and jet and put code in there that works, 416 00:21:36,920 --> 00:21:38,960 Speaker 1: that does the thing that the library is supposed to do, 417 00:21:39,080 --> 00:21:44,480 Speaker 1: but also retrieves, you know, all of the secrets that 418 00:21:44,520 --> 00:21:47,360 Speaker 1: are in your environment or like looks for and crypto 419 00:21:47,359 --> 00:21:50,960 Speaker 1: wallets or like production database passwords and stuff like that. 420 00:21:51,520 --> 00:21:54,160 Speaker 2: And if you're someone that can't read code or can't 421 00:21:54,200 --> 00:21:56,399 Speaker 2: read that kind of code, you would have no idea 422 00:21:56,400 --> 00:21:57,080 Speaker 2: this is happening. 423 00:21:57,440 --> 00:22:00,200 Speaker 1: You'd have no idea. And even if you're somebody who 424 00:22:00,240 --> 00:22:03,240 Speaker 1: does read code really well, if you you know, look 425 00:22:03,280 --> 00:22:06,399 Speaker 1: at that he said, it's it's important left You're like, 426 00:22:06,480 --> 00:22:08,800 Speaker 1: that sounds right, that looks right, like unless you are 427 00:22:08,840 --> 00:22:11,719 Speaker 1: really familiar with this library, and even if you are familiar, 428 00:22:11,920 --> 00:22:14,560 Speaker 1: you might just like gloss right over it. So it's 429 00:22:14,600 --> 00:22:17,560 Speaker 1: it's really really really dangerous, and like, this is the 430 00:22:17,600 --> 00:22:22,760 Speaker 1: type of thing that could you know, like maybe if 431 00:22:23,080 --> 00:22:25,720 Speaker 1: like if the LM is making you even twice as productive, 432 00:22:26,080 --> 00:22:28,000 Speaker 1: you know, that doesn't mean much if you know, there's 433 00:22:28,040 --> 00:22:31,159 Speaker 1: a chance you could destroy your entire company with a 434 00:22:31,160 --> 00:22:34,720 Speaker 1: catastrophic security breach, you know, yeah, leak all your database 435 00:22:34,760 --> 00:22:36,040 Speaker 1: to this this hacker. 436 00:22:37,600 --> 00:22:42,600 Speaker 3: And so yeah, not good. Yeah, is that becoming preblem? 437 00:22:43,359 --> 00:22:46,440 Speaker 1: I haven't heard much of it like happening in the wild, 438 00:22:46,800 --> 00:22:49,040 Speaker 1: but it's it's just one of those things that is 439 00:22:49,520 --> 00:22:52,760 Speaker 1: bound to happen because again, these are just just statistical 440 00:22:52,760 --> 00:22:55,119 Speaker 1: models that they don't they don't have the ability to 441 00:22:55,160 --> 00:22:58,480 Speaker 1: really reason about the actual nature of the things that 442 00:22:58,480 --> 00:23:00,600 Speaker 1: they're doing. They can try, you know, they can make 443 00:23:00,640 --> 00:23:03,359 Speaker 1: a sub LLM call and ask the other element like 444 00:23:03,640 --> 00:23:06,639 Speaker 1: does this look right? But then you're you know, burning 445 00:23:06,640 --> 00:23:07,040 Speaker 1: more and more. 446 00:23:07,480 --> 00:23:09,760 Speaker 2: And I also, at some point you are trusting the 447 00:23:09,840 --> 00:23:12,800 Speaker 2: statistical model to measure a statistical model's ability to do 448 00:23:12,920 --> 00:23:13,320 Speaker 2: is job. 449 00:23:13,880 --> 00:23:18,400 Speaker 1: Yes, exactly, And you know it kind of devolves. Anybody 450 00:23:18,400 --> 00:23:22,320 Speaker 1: who's who's used an LLM for coding knows that the 451 00:23:22,400 --> 00:23:25,480 Speaker 1: deeper you go into like a single prompt, like the 452 00:23:25,560 --> 00:23:28,440 Speaker 1: more back and forth, the larger the context window, the 453 00:23:29,000 --> 00:23:31,800 Speaker 1: more garbage it gets. And so like as you have 454 00:23:31,920 --> 00:23:35,760 Speaker 1: things like working off of other LLLM input, which is 455 00:23:35,800 --> 00:23:37,720 Speaker 1: effectively what you're doing in a large context window. You know, 456 00:23:37,720 --> 00:23:41,399 Speaker 1: it's just what the LM is sort of reprocessing the 457 00:23:41,440 --> 00:23:44,640 Speaker 1: text that it generated and that you've added to it. 458 00:23:44,640 --> 00:23:47,240 Speaker 1: It steadily gets worse the later in the context window. 459 00:23:47,280 --> 00:23:49,720 Speaker 1: So so basically all of these sort of mitigations are 460 00:23:50,400 --> 00:23:55,199 Speaker 1: you know, they've made surprising progress with the way that like, 461 00:23:55,640 --> 00:23:57,960 Speaker 1: you know, things like this don't happen is like just 462 00:23:58,160 --> 00:24:01,920 Speaker 1: raw hallucinations of like I think this library exists. They 463 00:24:01,920 --> 00:24:04,040 Speaker 1: happen a lot less now than they used to, but 464 00:24:04,080 --> 00:24:07,160 Speaker 1: they're they're just always they're always. 465 00:24:06,840 --> 00:24:09,280 Speaker 3: Gonnask and they're also always gonna be there. 466 00:24:09,600 --> 00:24:12,280 Speaker 2: Yeah, it's not it's not really something you could unless 467 00:24:12,280 --> 00:24:14,639 Speaker 2: we invent new maths. 468 00:24:15,040 --> 00:24:18,320 Speaker 1: Yeah, I mean, at least with the way we approach 469 00:24:18,440 --> 00:24:22,199 Speaker 1: AI right now, which is is is based purely on 470 00:24:22,880 --> 00:24:27,639 Speaker 1: language as tokens and it can't really fundamentally like understand 471 00:24:27,640 --> 00:24:29,960 Speaker 1: things outside of you know, word probabilities. 472 00:24:44,440 --> 00:24:45,959 Speaker 3: So where is this pressure coming from? 473 00:24:46,040 --> 00:24:49,320 Speaker 2: Because it feels like it's everywhere and you've got people 474 00:24:49,440 --> 00:24:52,399 Speaker 2: up Paul Graham who are wanking on about Oh that 475 00:24:52,760 --> 00:24:55,639 Speaker 2: I met a guy who writes ten thousand lines of code. 476 00:24:55,680 --> 00:24:57,359 Speaker 3: I think he said, what is it? 477 00:24:57,600 --> 00:24:59,520 Speaker 2: Are we just finding out how many people don't know 478 00:24:59,520 --> 00:25:00,000 Speaker 2: how code is? 479 00:25:00,119 --> 00:25:04,960 Speaker 1: Wooks? I think a lot of that. Yes, Like I said, 480 00:25:04,960 --> 00:25:07,880 Speaker 1: it's exactly the same way as like people, you know, 481 00:25:08,080 --> 00:25:10,040 Speaker 1: when the first image generation models were like, oh great, 482 00:25:10,040 --> 00:25:12,679 Speaker 1: we don't need graphic designers anymore, and then they are, 483 00:25:12,880 --> 00:25:15,600 Speaker 1: you know, oh great, we don't need customer support chat anymore, 484 00:25:15,720 --> 00:25:18,919 Speaker 1: because they fundamentally don't understand what those roles do. You know, 485 00:25:18,960 --> 00:25:23,399 Speaker 1: they're they think graphic designer and they think image generator. 486 00:25:23,920 --> 00:25:26,960 Speaker 1: But a graphic designer is a human being. That's that's 487 00:25:27,160 --> 00:25:30,520 Speaker 1: dealing with different stakeholders, dealing with people saying like no, no, no, 488 00:25:30,560 --> 00:25:33,680 Speaker 1: the logo can't have that, or you know, yes the logo. 489 00:25:33,520 --> 00:25:34,280 Speaker 3: Must get bump. 490 00:25:35,200 --> 00:25:38,920 Speaker 1: Yeah, yeah, exactly, and they're dealing with you know, different requirements, 491 00:25:38,960 --> 00:25:41,760 Speaker 1: and then they're needing to make variations, which is something 492 00:25:41,760 --> 00:25:45,080 Speaker 1: that LMS are not always very good at, or I 493 00:25:45,080 --> 00:25:48,440 Speaker 1: should just say jeneratorf AI is not always very good at. Yeah, 494 00:25:49,320 --> 00:25:52,560 Speaker 1: and so so I strange part of it is is 495 00:25:52,560 --> 00:25:54,359 Speaker 1: is people who you know, like I said, you know, 496 00:25:54,400 --> 00:25:56,520 Speaker 1: you get these like brief bursts where you're like, oh 497 00:25:56,560 --> 00:25:59,040 Speaker 1: my god, I just say so much time and you 498 00:25:59,160 --> 00:26:01,800 Speaker 1: extrapolate that. So some of it I think as engineers 499 00:26:01,840 --> 00:26:04,600 Speaker 1: who are just like actual engineers who know how to code, 500 00:26:04,600 --> 00:26:06,880 Speaker 1: who see these things and they think like I did 501 00:26:06,880 --> 00:26:10,320 Speaker 1: this today, and like I must have been so much 502 00:26:10,359 --> 00:26:12,520 Speaker 1: more productive as a result. You know, I saw this 503 00:26:12,560 --> 00:26:16,160 Speaker 1: one thing happen. But they don't really actually measure in 504 00:26:16,280 --> 00:26:19,359 Speaker 1: depth what they produced and was it more than what 505 00:26:19,400 --> 00:26:22,520 Speaker 1: they would have produced. There have been some studies to 506 00:26:22,560 --> 00:26:25,800 Speaker 1: measure that, and they haven't looked particularly good for lllms. 507 00:26:25,840 --> 00:26:29,640 Speaker 1: You know, if you actually compare people, you know, using 508 00:26:29,680 --> 00:26:34,120 Speaker 1: AI versus not using AI, the results don't always look 509 00:26:34,160 --> 00:26:38,399 Speaker 1: particularly great for AI. And one thing that's very common 510 00:26:38,400 --> 00:26:40,720 Speaker 1: out of that is that people overestimate their performance. 511 00:26:41,160 --> 00:26:43,320 Speaker 2: So I think that might be a problem across or 512 00:26:43,480 --> 00:26:46,399 Speaker 2: like a lot of people don't. I also think my 513 00:26:46,440 --> 00:26:49,480 Speaker 2: grander theory with all of this is a lot of 514 00:26:49,480 --> 00:26:50,800 Speaker 2: people don't know what work is. 515 00:26:51,280 --> 00:26:52,040 Speaker 3: Like a lot of these. 516 00:26:51,880 --> 00:26:54,399 Speaker 2: Investors and managers and such, and even people in the 517 00:26:54,440 --> 00:26:57,439 Speaker 2: media don't seem to actually know what jobs are and 518 00:26:57,480 --> 00:26:59,439 Speaker 2: how the jobs work, and they think that things like 519 00:26:59,480 --> 00:27:02,400 Speaker 2: coding is just like I said earlier, yeah, just walk 520 00:27:02,400 --> 00:27:04,600 Speaker 2: into work or write ten thousand lines of code on 521 00:27:04,640 --> 00:27:07,199 Speaker 2: a walk home. But now I can write twenty billion 522 00:27:07,240 --> 00:27:09,280 Speaker 2: lines of code because that's all my job is. 523 00:27:10,359 --> 00:27:12,800 Speaker 1: Yeah, absolutely, I mean this is this is something I 524 00:27:12,840 --> 00:27:15,159 Speaker 1: talk about in my article, is that there's always this 525 00:27:15,240 --> 00:27:17,880 Speaker 1: like degree of separation and the people who are talking 526 00:27:17,960 --> 00:27:23,080 Speaker 1: the most about AI coding' aren't really coders and they're 527 00:27:23,119 --> 00:27:26,280 Speaker 1: not really providing like detailed reproduction steps. 528 00:27:27,600 --> 00:27:27,760 Speaker 4: You know. 529 00:27:27,840 --> 00:27:31,520 Speaker 1: I know engineers who love using AI like every single day, 530 00:27:31,560 --> 00:27:33,320 Speaker 1: and they use it for like all of their projects. 531 00:27:33,600 --> 00:27:35,360 Speaker 1: And I know really good ones to do that. And 532 00:27:35,600 --> 00:27:39,080 Speaker 1: if I asked them like, hey, how could I like 533 00:27:39,160 --> 00:27:42,760 Speaker 1: be more likely be more like you? How could I 534 00:27:42,840 --> 00:27:45,640 Speaker 1: be a better coder? One of the last things they'll 535 00:27:45,640 --> 00:27:48,359 Speaker 1: say is, probably, you know, start using AI more, for 536 00:27:48,760 --> 00:27:52,320 Speaker 1: it's really just a tool to accomplish part of their job. 537 00:27:52,520 --> 00:27:54,640 Speaker 1: And so so yeah, I think there's a lot of 538 00:27:55,119 --> 00:27:59,800 Speaker 1: you know, there's probably you know, plenty of genuine people 539 00:27:59,840 --> 00:28:02,639 Speaker 1: who who just like you know, they've never written a 540 00:28:02,640 --> 00:28:04,560 Speaker 1: line of code in their life. They pull up Lovable 541 00:28:04,880 --> 00:28:06,840 Speaker 1: they say, generate me an app that does this, and 542 00:28:06,880 --> 00:28:09,240 Speaker 1: like they legitimately are like, oh my gosh, it actually 543 00:28:09,280 --> 00:28:14,120 Speaker 1: worked like I I can code, and they just yeah, 544 00:28:14,119 --> 00:28:16,480 Speaker 1: they just you know, naturally, you know, they don't realize it. 545 00:28:16,880 --> 00:28:20,120 Speaker 1: They don't realize that, like there is so much more 546 00:28:20,160 --> 00:28:25,720 Speaker 1: to this actual practice, and you know they're they're they're 547 00:28:25,760 --> 00:28:28,159 Speaker 1: just not in tune with the way that coding actually works. 548 00:28:29,280 --> 00:28:32,399 Speaker 2: Yeah, it's a little bit sad as well, because it 549 00:28:32,440 --> 00:28:34,840 Speaker 2: really feels like a lot of this is just the 550 00:28:35,480 --> 00:28:37,280 Speaker 2: point you've made about like the image generators. 551 00:28:37,359 --> 00:28:40,000 Speaker 3: It's just this immediate moment of wow, look what this 552 00:28:40,040 --> 00:28:40,360 Speaker 3: could do. 553 00:28:40,520 --> 00:28:42,360 Speaker 2: Imagine what it could do next, and then you look 554 00:28:42,400 --> 00:28:44,600 Speaker 2: at what it could do next, and it can't, Like 555 00:28:44,680 --> 00:28:47,240 Speaker 2: it looks like it can generate code, but it can't 556 00:28:47,280 --> 00:28:50,240 Speaker 2: actually generate software like it cut. 557 00:28:50,520 --> 00:28:52,320 Speaker 3: It doesn't seem to do the steps. 558 00:28:51,920 --> 00:28:56,600 Speaker 2: That make software functional and scale because there's these tendrils 559 00:28:58,040 --> 00:29:01,160 Speaker 2: of from software into the infrastructure to make sure it 560 00:29:01,200 --> 00:29:04,160 Speaker 2: can be shown in different places or to make sure 561 00:29:04,200 --> 00:29:06,440 Speaker 2: that it actually functions. 562 00:29:05,920 --> 00:29:11,160 Speaker 3: On a day to day basic Yeah, and it's like, yeah. 563 00:29:10,320 --> 00:29:12,760 Speaker 1: It's a pretty simple like curve, Like you know, you 564 00:29:12,840 --> 00:29:16,160 Speaker 1: start out and it generates so much code that's like 565 00:29:16,680 --> 00:29:20,000 Speaker 1: pretty correct, really fast. When you if you start a 566 00:29:20,040 --> 00:29:25,000 Speaker 1: completely bare project from just like a lovable prompt or 567 00:29:25,040 --> 00:29:28,959 Speaker 1: something like that. But as you go on, you know, 568 00:29:29,280 --> 00:29:31,440 Speaker 1: it's like the curve kind of flattens and it goes 569 00:29:31,480 --> 00:29:33,880 Speaker 1: down and it becomes less and less productive, and I 570 00:29:33,880 --> 00:29:37,280 Speaker 1: think eventually, usually like pure vibe coded projects hit a 571 00:29:37,280 --> 00:29:41,960 Speaker 1: pretty big wall because you're just introducing so much code 572 00:29:42,480 --> 00:29:47,560 Speaker 1: and it's not consistent, it's not it's not using shared tools, 573 00:29:47,600 --> 00:29:50,000 Speaker 1: and so eventually you just end up with this you know, 574 00:29:50,040 --> 00:29:52,000 Speaker 1: they call it spaghetti code. It's you know, code that 575 00:29:52,440 --> 00:29:56,520 Speaker 1: is so interwoven and difficult to understand that like you 576 00:29:56,560 --> 00:29:58,440 Speaker 1: can't actually see what's going on with it. 577 00:29:59,200 --> 00:30:01,920 Speaker 2: Yeah, it was like context is the whole problem, just 578 00:30:01,960 --> 00:30:06,040 Speaker 2: because even if not to say, like LLM generated writing 579 00:30:06,160 --> 00:30:09,000 Speaker 2: is dog shit, and I think it's worse than code 580 00:30:09,040 --> 00:30:13,000 Speaker 2: because code code, code is functional in a way that 581 00:30:13,040 --> 00:30:15,920 Speaker 2: I don't think writing has to be. Like writing conveys meaning. 582 00:30:16,000 --> 00:30:19,520 Speaker 2: But good writing is usually more than just I am 583 00:30:19,680 --> 00:30:22,800 Speaker 2: entering this writing into someone's brain. So something happens in 584 00:30:22,800 --> 00:30:25,520 Speaker 2: the way that code is, but writing it they share 585 00:30:25,520 --> 00:30:28,480 Speaker 2: the same problem, which is great writing has contextual awareness, 586 00:30:28,520 --> 00:30:31,440 Speaker 2: It builds, it connects, there is an argument or there 587 00:30:31,480 --> 00:30:34,120 Speaker 2: is an evocation from it. In the case with software, 588 00:30:34,160 --> 00:30:37,320 Speaker 2: it appears to be. If you don't know every reason 589 00:30:37,360 --> 00:30:40,240 Speaker 2: that everything was done and fully understand the reasons that 590 00:30:40,280 --> 00:30:43,200 Speaker 2: were previous and the reasons that are happening right now, 591 00:30:43,640 --> 00:30:46,840 Speaker 2: you kind of will fuck something up naturally, even if 592 00:30:46,840 --> 00:30:48,760 Speaker 2: you do know how to code, if you just don't 593 00:30:48,800 --> 00:30:51,680 Speaker 2: read any of the notes, if you if if it's 594 00:30:51,720 --> 00:30:54,680 Speaker 2: not clear why things were done, things will break anyway, right. 595 00:30:55,000 --> 00:30:58,200 Speaker 1: Yeah, exactly. And great writing, you know, it knows when 596 00:30:58,280 --> 00:31:00,760 Speaker 1: to you know, you know, a great knows you know, 597 00:31:00,800 --> 00:31:02,680 Speaker 1: when they need to explain something, when they don't need 598 00:31:02,720 --> 00:31:05,000 Speaker 1: to explain something. You know, they know, you know, I'm 599 00:31:05,040 --> 00:31:07,520 Speaker 1: writing a trade publication like I don't need to explain 600 00:31:07,640 --> 00:31:10,120 Speaker 1: how friction works, or I'm writing, you know, a public 601 00:31:10,160 --> 00:31:12,680 Speaker 1: press release, Okay, I do need to explain how friction works, 602 00:31:12,720 --> 00:31:17,880 Speaker 1: or you know whatever. So it's it's very you know, 603 00:31:18,720 --> 00:31:21,280 Speaker 1: these are the things that the lms are like, yeah, exactly, 604 00:31:21,360 --> 00:31:23,320 Speaker 1: just like very not good at you know, they can 605 00:31:23,400 --> 00:31:26,920 Speaker 1: they can generate stuff that looks good, but it's just 606 00:31:27,000 --> 00:31:28,800 Speaker 1: the more you try to build on top of it, 607 00:31:29,120 --> 00:31:31,200 Speaker 1: the more it'll it'll end up restating. Like you know, 608 00:31:32,480 --> 00:31:35,800 Speaker 1: lms are really good at writing, Like the classic school 609 00:31:35,960 --> 00:31:41,960 Speaker 1: level five paragraph essay. But everybody who actually writes anything 610 00:31:42,000 --> 00:31:44,760 Speaker 1: at all knows that, like the five paragraphed essays that 611 00:31:44,800 --> 00:31:46,680 Speaker 1: you wrote in high school are like terrible and like 612 00:31:46,840 --> 00:31:49,680 Speaker 1: nobody wants to read something structured as like you know, 613 00:31:50,240 --> 00:31:53,680 Speaker 1: premise three, argumented a paragraphs conclusion like that's it's really 614 00:31:53,760 --> 00:32:00,760 Speaker 1: bad like unconvincing writing. Uh. And it's the same with 615 00:32:00,880 --> 00:32:02,960 Speaker 1: you know, lms are really good at making toys and 616 00:32:03,000 --> 00:32:06,000 Speaker 1: like like quick things that like are are fun or 617 00:32:06,040 --> 00:32:08,320 Speaker 1: you know, maybe a little bit useful in certain situations, 618 00:32:09,160 --> 00:32:11,800 Speaker 1: but really bad at writing things that you know, uh, 619 00:32:12,560 --> 00:32:15,160 Speaker 1: like like a book level type things like you know, 620 00:32:15,280 --> 00:32:18,120 Speaker 1: ais are horrible at writing books because they restate things 621 00:32:18,160 --> 00:32:20,840 Speaker 1: and they lose track of what they're talking about. 622 00:32:20,880 --> 00:32:23,560 Speaker 2: And the more stuff it creates, right, the more it 623 00:32:23,640 --> 00:32:25,600 Speaker 2: looks over, the more confused it gets. 624 00:32:26,080 --> 00:32:30,280 Speaker 1: Exactly It's it's really really similar. I think that coding 625 00:32:30,280 --> 00:32:33,080 Speaker 1: has really just followed the same trajectory as like all 626 00:32:33,120 --> 00:32:34,920 Speaker 1: of these other things where we're like, oh, we don't 627 00:32:34,920 --> 00:32:38,920 Speaker 1: need copywriters anymore. Oh, we don't need uh designers anymore. 628 00:32:38,960 --> 00:32:41,120 Speaker 1: We don't need graphic designers, we don't need this, that 629 00:32:41,200 --> 00:32:43,760 Speaker 1: and the other. We don't need lawyers anymore we have 630 00:32:43,840 --> 00:32:46,920 Speaker 1: an ll m P or a lawyer, and you just 631 00:32:47,000 --> 00:32:50,440 Speaker 1: very quickly realize that like these these jobs aren't uh 632 00:32:51,000 --> 00:32:56,640 Speaker 1: you know, dumb factory uh like producing you know, pulling 633 00:32:56,640 --> 00:32:59,960 Speaker 1: a lever type jobs. They're they're about interaction. 634 00:32:59,640 --> 00:33:02,480 Speaker 2: And that's an insult to factory labor. That is very 635 00:33:02,480 --> 00:33:05,640 Speaker 2: hard work. But it's not like a repetitive action that 636 00:33:05,760 --> 00:33:06,920 Speaker 2: is always the same thing. 637 00:33:07,880 --> 00:33:10,160 Speaker 1: No, not in a sense anyway. Yeah, and like a 638 00:33:10,200 --> 00:33:14,480 Speaker 1: factory worker, you know, is doing a lot more than 639 00:33:14,520 --> 00:33:15,400 Speaker 1: just pulling levers. 640 00:33:15,680 --> 00:33:18,120 Speaker 2: Yeah, of course, but it's not just hitting a button. 641 00:33:18,240 --> 00:33:20,880 Speaker 2: But I think people condense coding to this thing. 642 00:33:22,040 --> 00:33:25,760 Speaker 1: Yeah, And it's very similar to like robotics in factories too, 643 00:33:25,800 --> 00:33:27,360 Speaker 1: where you know, the promise is like, oh, you know, 644 00:33:27,400 --> 00:33:28,800 Speaker 1: we'll just have a robot do this thing that a 645 00:33:28,840 --> 00:33:31,240 Speaker 1: human does. But the human is doing a lot more 646 00:33:31,280 --> 00:33:33,640 Speaker 1: than just you know, pulling the lever. They're like observing 647 00:33:33,680 --> 00:33:36,360 Speaker 1: the process. They're making sure that things are not getting 648 00:33:36,400 --> 00:33:38,479 Speaker 1: broken or getting gummied up and stuff like that. So 649 00:33:39,720 --> 00:33:42,840 Speaker 1: there's just there's just limits to what machines can do 650 00:33:43,080 --> 00:33:46,160 Speaker 1: when they're not actually intelligent. And that's just what it 651 00:33:46,200 --> 00:33:46,680 Speaker 1: comes down to. 652 00:33:47,520 --> 00:33:50,560 Speaker 2: Do you buy the any of these companies like Google 653 00:33:50,800 --> 00:33:52,960 Speaker 2: a writing thirty percent of their code with AI. 654 00:33:57,360 --> 00:33:59,760 Speaker 1: The thing about like those numbers is that they are 655 00:33:59,840 --> 00:34:03,560 Speaker 1: soon easy, maybe not to game, but to First of all, 656 00:34:03,560 --> 00:34:06,239 Speaker 1: I don't know anyone who's actually like measuring this in 657 00:34:06,280 --> 00:34:11,399 Speaker 1: a really like effective way, because the thing about your 658 00:34:11,440 --> 00:34:15,240 Speaker 1: like coding editor is that it's a lot of people 659 00:34:15,320 --> 00:34:17,759 Speaker 1: I don't like to use the AI autocompletion, but some 660 00:34:17,800 --> 00:34:20,319 Speaker 1: people do. You know, there's there's pieces where it'll you know, 661 00:34:20,400 --> 00:34:23,440 Speaker 1: you'll start writing some boiler plate and it'll like pop 662 00:34:23,560 --> 00:34:25,439 Speaker 1: up you know what it thinks you mean to write, 663 00:34:25,440 --> 00:34:27,520 Speaker 1: and you'll just hit tab the tab key on your 664 00:34:27,560 --> 00:34:30,719 Speaker 1: keyboard and it'll just like finish the line of code 665 00:34:30,760 --> 00:34:34,960 Speaker 1: that you were writing. And maybe that would yeah, yeah, exactly, 666 00:34:35,120 --> 00:34:38,799 Speaker 1: and it might even very often what it produces is wrong, 667 00:34:38,880 --> 00:34:41,000 Speaker 1: but it's like seventy five percent, right, So you're like, oh, 668 00:34:41,040 --> 00:34:43,719 Speaker 1: I can just like use these keystrokes, I can save 669 00:34:43,760 --> 00:34:46,120 Speaker 1: these keystrokes, and I can just fix what it did wrong. 670 00:34:47,560 --> 00:34:51,440 Speaker 2: Oh God, it's like saying auto correct wrote parts of 671 00:34:51,480 --> 00:34:51,960 Speaker 2: your book. 672 00:34:52,560 --> 00:34:57,720 Speaker 1: Yeah, it really is. And you know there are times 673 00:34:57,760 --> 00:35:02,160 Speaker 1: where like an AI does is fully write the code 674 00:35:02,160 --> 00:35:05,640 Speaker 1: for a feature, so people use you know things where 675 00:35:05,640 --> 00:35:10,880 Speaker 1: they can prompt an LLM and it will with like 676 00:35:10,960 --> 00:35:12,719 Speaker 1: what they want the feature to be or what they 677 00:35:12,719 --> 00:35:15,279 Speaker 1: want the bug fix to be, and it will go 678 00:35:15,320 --> 00:35:17,480 Speaker 1: and we'll write all the code and it will make 679 00:35:17,520 --> 00:35:19,759 Speaker 1: the sort of we call it a merge request, but 680 00:35:19,800 --> 00:35:25,400 Speaker 1: that's the request for new code that then gets reviewed. 681 00:35:25,520 --> 00:35:27,680 Speaker 1: It will go all that way. But the thing is, 682 00:35:28,040 --> 00:35:31,320 Speaker 1: you know, it might have written the code, but somebody 683 00:35:31,800 --> 00:35:36,799 Speaker 1: took time away from you know, their normal job, which 684 00:35:36,840 --> 00:35:39,800 Speaker 1: just be writing code, to write a really good prompt 685 00:35:40,239 --> 00:35:41,759 Speaker 1: to make sure it didn't screw it up, and then 686 00:35:41,800 --> 00:35:45,480 Speaker 1: reinteract with it. And so you know, did it write 687 00:35:45,520 --> 00:35:50,320 Speaker 1: the code, yes, but did it do the task not really, 688 00:35:50,360 --> 00:35:52,640 Speaker 1: because it needed somebody else to do some support work 689 00:35:52,640 --> 00:35:54,160 Speaker 1: to make it even possible for it to do it. 690 00:35:54,200 --> 00:35:56,480 Speaker 1: And that person could have That person needed to be 691 00:35:56,520 --> 00:35:58,359 Speaker 1: technical and needed to be able to say like, oh, 692 00:35:58,400 --> 00:35:59,879 Speaker 1: you need to look in this part of the code base, 693 00:36:01,840 --> 00:36:06,439 Speaker 1: And so you end up just getting the same type 694 00:36:06,440 --> 00:36:11,000 Speaker 1: of like actual work from the actual specialist who knows 695 00:36:11,000 --> 00:36:13,239 Speaker 1: how to code the same amount of work. They're just 696 00:36:13,320 --> 00:36:14,480 Speaker 1: doing something slightly different. 697 00:36:15,800 --> 00:36:18,160 Speaker 2: Cole has been such a pleasure having you here. Where 698 00:36:18,200 --> 00:36:19,000 Speaker 2: can people find you? 699 00:36:19,680 --> 00:36:23,960 Speaker 1: For sure. My blog is Coulson dot dev c O 700 00:36:24,080 --> 00:36:26,560 Speaker 1: L t O N dot dev. I don't post that 701 00:36:26,640 --> 00:36:30,719 Speaker 1: often because I work full time and I just I 702 00:36:30,800 --> 00:36:33,239 Speaker 1: just post when something really gets to me. But you know, 703 00:36:33,320 --> 00:36:35,560 Speaker 1: I might. I might say some things here and there. 704 00:36:35,719 --> 00:36:37,400 Speaker 3: And you have your excellent blog that I brought you 705 00:36:37,440 --> 00:36:37,719 Speaker 3: on for. 706 00:36:38,080 --> 00:36:40,160 Speaker 1: Yeah, yeah, that's my that's my most recent one. You 707 00:36:40,160 --> 00:36:41,680 Speaker 1: can feel free to check it out. I'm sure the 708 00:36:41,719 --> 00:36:43,879 Speaker 1: link to that will be in the description. But yeah, 709 00:36:43,880 --> 00:36:44,680 Speaker 1: it was great being here. 710 00:36:45,040 --> 00:36:56,000 Speaker 5: Thanks so much, Thank you for listening to Better Offline. 711 00:36:56,120 --> 00:36:58,560 Speaker 6: The editor and composer of the Better Offline theme song 712 00:36:58,640 --> 00:37:01,239 Speaker 6: is Matosowski. You can check out more of his music 713 00:37:01,320 --> 00:37:04,959 Speaker 6: and audio projects at Mattasowski dot com m A T 714 00:37:04,960 --> 00:37:09,400 Speaker 6: t O s O W s ki dot com. You 715 00:37:09,440 --> 00:37:11,960 Speaker 6: can email me at easy at Better offline dot com 716 00:37:12,040 --> 00:37:14,320 Speaker 6: or visit better offline dot com to find more podcast 717 00:37:14,400 --> 00:37:17,720 Speaker 6: links and of course, my newsletter. I also really recommend 718 00:37:17,760 --> 00:37:19,680 Speaker 6: you go to chat dot Where's youreaed dot at to 719 00:37:19,760 --> 00:37:22,080 Speaker 6: visit the discord, and go to our slash. 720 00:37:21,840 --> 00:37:25,000 Speaker 5: Better Offline to check out our reddit. Thank you so 721 00:37:25,080 --> 00:37:28,520 Speaker 5: much for listening. Better Offline is a production of cool 722 00:37:28,600 --> 00:37:30,600 Speaker 5: Zone Media. For more from cool Zone, Media. 723 00:37:30,960 --> 00:37:34,120 Speaker 4: Visit our website cool zonemedia dot com, or check us 724 00:37:34,120 --> 00:37:37,120 Speaker 4: out on the iHeartRadio app, Apple Podcasts, or wherever you 725 00:37:37,160 --> 00:37:38,280 Speaker 4: get your podcasts.