WEBVTT - AI Isn't Making Engineers 10x As Productive with Colton Voege 0:00:02.440 --> 0:00:03.800 Also media. 0:00:05.600 --> 0:00:07.760 Hello and welcome to Better Offline. I'm your host ed 0:00:07.880 --> 0:00:09.600 zitron as eva by our merchandise. 0:00:09.640 --> 0:00:10.760 Subscribe to the newsletter. 0:00:10.760 --> 0:00:23.000 It's all in the episode notes. 0:00:24.520 --> 0:00:27.840 And today I'm joined by programmer Colton Bogie, who wrote 0:00:27.880 --> 0:00:30.360 an excellent piece about a month ago called no AI 0:00:30.480 --> 0:00:32.519 is not making engineers ten X is powerful. 0:00:33.479 --> 0:00:34.720 Colp. Thanks for joining me. 0:00:34.920 --> 0:00:35.680 Thank you for having me. 0:00:36.360 --> 0:00:38.280 So tell me a little bit about your day to 0:00:38.360 --> 0:00:40.160 day work. What do you do for a living. 0:00:41.240 --> 0:00:45.200 Yeah, so I'm a software engineer, and I work specifically 0:00:45.200 --> 0:00:48.760 in what's called web application development. So that's kind of 0:00:48.800 --> 0:00:51.760 like building rich applications that work in the web browser. 0:00:51.840 --> 0:00:55.240 So think of something like Amazon or Google Drive or 0:00:55.240 --> 0:00:57.080 something where you know you can do a lot of 0:00:57.080 --> 0:01:01.840 interactive features within the applihation within a browser, right, So. 0:01:01.840 --> 0:01:04.360 Kind of like the fundament of most cloud software. 0:01:04.800 --> 0:01:08.160 Yeah. I would say that the if not the majority 0:01:08.200 --> 0:01:10.959 of engineers work in web application development, then probably the 0:01:11.000 --> 0:01:12.440 plurality do right now. 0:01:12.480 --> 0:01:16.400 And it's also a large amount of how people interact 0:01:16.440 --> 0:01:18.080 with software is now in this way. 0:01:17.880 --> 0:01:22.119 Though, Yeah, exactly. Yeah, and that's why that's why most 0:01:22.120 --> 0:01:24.480 developers perhaps work in this field now. 0:01:25.160 --> 0:01:27.880 And you'd think something like that would be perfect for 0:01:27.920 --> 0:01:28.640 AI coding. 0:01:29.720 --> 0:01:29.959 It is. 0:01:30.959 --> 0:01:35.200 It is definitely the most applicable place for AI coding 0:01:35.280 --> 0:01:38.399 of all of them. There's sort of a there's sort 0:01:38.440 --> 0:01:41.119 of a thing where people are like, you know, oh, 0:01:41.240 --> 0:01:43.639 you know, lms don't really work really well for my language, 0:01:43.640 --> 0:01:45.440 and they'll be talking about something like RUST, which is 0:01:45.480 --> 0:01:49.120 more of like a a high performance language rather than 0:01:49.120 --> 0:01:50.280 a web language. 0:01:51.840 --> 0:01:54.200 What is a hypophobic language in this case. 0:01:54.800 --> 0:01:59.320 Just something that's designed to work on like weak hardware 0:01:59.480 --> 0:02:04.080 or a like extremely high speed So like for example, 0:02:04.120 --> 0:02:06.240 like video games aren't built in RUSS. They're usually built 0:02:06.240 --> 0:02:08.320 in a language called C plus plus, but they're they're 0:02:08.320 --> 0:02:12.560 built in a high performance language because you know, you're 0:02:12.880 --> 0:02:15.080 trying to max out as much as you can do. 0:02:15.160 --> 0:02:18.440 Ye're taking advantage of the high performance compute you have. 0:02:18.720 --> 0:02:21.360 Yeah, whereas with a website like you know, like Amazon 0:02:21.440 --> 0:02:24.840 or something like that, you're really not pushing the limit 0:02:24.919 --> 0:02:30.000 in terms of interactivity, and so more modest languages are fine. 0:02:30.760 --> 0:02:31.200 Got it? 0:02:31.280 --> 0:02:33.280 So you laid it out really nicely in the piece, 0:02:33.320 --> 0:02:35.399 so you can have to repeat a few things, I imagine, 0:02:35.480 --> 0:02:40.480 but run me through why AI coding tools can't actually 0:02:40.520 --> 0:02:44.760 make you a ten x engineer or ten times better engineer. 0:02:46.760 --> 0:02:52.200 Yeah, so basically AI is really good at AI is 0:02:52.240 --> 0:02:56.400 almost shockingly good at generating code, and it's almost shockingly 0:02:56.400 --> 0:03:03.000 good at generating code that runs, and it can answer 0:03:03.040 --> 0:03:05.520 a lot of questions that are that are like difficult 0:03:06.360 --> 0:03:09.959 or at least, you know, annoying is probably the better word. 0:03:09.960 --> 0:03:11.680 It's really good at dealing with things that are annoying. 0:03:12.040 --> 0:03:14.680 It's really good at, you know, so encoding. We have 0:03:14.720 --> 0:03:18.079 this concept called boilerplate, and it's it's just code that 0:03:18.120 --> 0:03:20.560 you have to kind of rewrite a lot. And as 0:03:20.600 --> 0:03:23.240 an engineer, ideally you don't have to write a lot 0:03:23.240 --> 0:03:26.240 of boilerplate. Ideal you automate things and distract things, so 0:03:26.320 --> 0:03:28.920 you you don't need to do that very often. But 0:03:29.080 --> 0:03:31.640 it's sort of like a requirement of the job. And 0:03:31.720 --> 0:03:34.359 it's really good at writing that because you know, it's 0:03:34.480 --> 0:03:38.520 just it's kind of like low intent, high volume code, 0:03:39.720 --> 0:03:43.280 quantity over quality type thing. So it's really good. It's 0:03:43.320 --> 0:03:47.680 stuff like that. The problem is that generating code really 0:03:47.760 --> 0:03:52.120 isn't the hard part of being a software engineer. It's 0:03:52.160 --> 0:03:54.840 it's one of the things that really matters, and it's like, 0:03:54.880 --> 0:03:57.480 obviously if you go to college to learn how to code, 0:03:57.480 --> 0:03:58.960 what you're going to spend most of the time doing 0:03:59.040 --> 0:04:02.480 is is typing code, but it's it's really just like 0:04:02.600 --> 0:04:06.000 one thing that you're doing, Like in reality, you're doing 0:04:06.000 --> 0:04:07.880 a lot more thinking about like, Okay, how does this 0:04:08.120 --> 0:04:10.520 work with the systems already have? How do I avoid 0:04:11.160 --> 0:04:14.400 creating what's called tech debt? So tech debt is basically 0:04:14.440 --> 0:04:16.440 like an easy way of thinking about it is when 0:04:16.480 --> 0:04:19.360 you write code that makes writing future code harder. So 0:04:19.480 --> 0:04:22.720 an example would be, you know, you want to write 0:04:22.920 --> 0:04:25.799 like a piece of logic that, let's say computes sales tax. 0:04:25.839 --> 0:04:27.640 If you're, you know, making an online story, you'd want 0:04:27.640 --> 0:04:30.080 to compute sales tax based on where they are. You 0:04:30.120 --> 0:04:31.719 only want to write that once. You don't want to 0:04:31.720 --> 0:04:34.839 have two different places that you know handle sales tax, 0:04:34.880 --> 0:04:38.520 because then, if you know, Illinois changes their sales tax, right, 0:04:38.520 --> 0:04:40.360 you have to change two places instead of one. 0:04:41.000 --> 0:04:44.039 It doesn't doesn't The very nature of code also mean 0:04:44.080 --> 0:04:46.400 that it naturally creates tech debt. 0:04:46.920 --> 0:04:49.920 You know, you can try to mitigate as much as 0:04:49.920 --> 0:04:52.120 you can, but there's not even an idea like that 0:04:52.680 --> 0:04:57.320 all code is tech debt. And and what LLLMS so 0:04:57.640 --> 0:04:59.880 kind of what I mentioned before. You know, lms are 0:05:00.080 --> 0:05:02.240 get at producing boilerplate. But if you get to the 0:05:02.279 --> 0:05:04.880 point where boilerplate is really easy to produce and you're 0:05:04.920 --> 0:05:08.280 not constantly thinking like, Okay, how do I avoid doing 0:05:08.320 --> 0:05:12.159 this in the future, you start, yeah, exactly, you start 0:05:12.200 --> 0:05:15.159 generating more and more code. And just like generating code 0:05:15.240 --> 0:05:17.480 is an it's kind of like writing in terms of, 0:05:17.520 --> 0:05:20.119 like you know, writing a book, like a good writer 0:05:20.800 --> 0:05:22.520 writes fewer words rather than more. 0:05:22.760 --> 0:05:24.400 Yeah. Yeah, brevity is the soul of wit. 0:05:24.800 --> 0:05:28.159 Now. It's because what I've been realizing is that there 0:05:28.200 --> 0:05:31.400 is this delta between people that actually write software and 0:05:31.440 --> 0:05:34.280 people that are excited about AI coding. Because I had 0:05:34.360 --> 0:05:36.919 Carl Brown from the Internet of Bugs on and it 0:05:37.000 --> 0:05:39.320 was this thing of yeah, being the software engineer is 0:05:39.360 --> 0:05:42.360 not just writing one hundred lines of code and then 0:05:42.720 --> 0:05:45.000 giving the thumbs up to your boss. You have to 0:05:45.080 --> 0:05:49.359 interact with various different parts of the organization. Early on 0:05:49.400 --> 0:05:50.920 in the piece as well, you had this bit called 0:05:50.920 --> 0:05:53.240 the math no link to this obviously in the episode notes, 0:05:53.240 --> 0:05:56.400 where it's any software engineer has worked on actual code 0:05:56.400 --> 0:05:59.560 in the natural company knows that you have these steps 0:06:00.120 --> 0:06:02.400 where it isn't just like icode and then I give 0:06:02.440 --> 0:06:03.960 the thumbs up and I'm dumb for the day. You 0:06:04.000 --> 0:06:06.400 have to go through a reviewer, you have to wait 0:06:06.440 --> 0:06:08.560 for them to get back to you. You have to 0:06:08.600 --> 0:06:10.919 contact switching as well. We have to change different windows 0:06:10.920 --> 0:06:13.520 and do different things. There's a shit ton of just 0:06:13.600 --> 0:06:17.599 intermediary work that is nothing to do with actually writing code, right. 0:06:18.720 --> 0:06:22.920 Yeah, absolutely, I mean it's it really is more of 0:06:23.240 --> 0:06:25.760 you know, there's parts of coding that is more science 0:06:25.760 --> 0:06:28.040 than there's parts of coding that are more art and 0:06:28.160 --> 0:06:30.760 more you know, in college, is really common if you're 0:06:30.760 --> 0:06:34.159 getting computer science deority to take communications classes because you 0:06:34.240 --> 0:06:35.840 just have to interact with a lot of people. So 0:06:36.120 --> 0:06:40.320 the standard way that a thing gets done in a 0:06:40.360 --> 0:06:43.640 software project is you have somebody called a product manager, 0:06:43.720 --> 0:06:47.200 and that person's that person's job is to think about 0:06:47.240 --> 0:06:50.440 what the product is as it exists now and what 0:06:50.480 --> 0:06:52.480 should be in the future, and like what features we 0:06:52.480 --> 0:06:55.240 should build, Like basically more than anything, it's you know, 0:06:55.279 --> 0:06:57.080 what what should we build next? And then you have 0:06:57.120 --> 0:06:59.160 designers who are going to decide how that thing should look. 0:06:59.520 --> 0:07:03.480 And engineers have to be this meeting influence because they're 0:07:03.520 --> 0:07:06.640 the only ones in the process who actually know what 0:07:06.680 --> 0:07:08.880 it's like, how hard it is going to be to 0:07:08.880 --> 0:07:12.040 build something. So a frequent interaction is like, you know, 0:07:12.080 --> 0:07:13.840 product maators are like, oh, we should add this, and 0:07:13.880 --> 0:07:16.440 an engineer has to kind of step in and be like, no, 0:07:16.560 --> 0:07:20.000 that's borderline impossible or you know that would take six months, 0:07:21.000 --> 0:07:23.000 right and you want coding. 0:07:23.120 --> 0:07:25.960 Lllms don't fix that problem. They don't do they actually 0:07:25.960 --> 0:07:27.960 make it easier to develop products. 0:07:28.200 --> 0:07:33.400 I think there are uses for LMS in coding, and. 0:07:33.440 --> 0:07:38.240 I'm not even denying that. I'm just like, how, like, 0:07:38.280 --> 0:07:41.280 what do what do those uses end up actually creating? 0:07:42.280 --> 0:07:46.200 Basically, you know, what I found is that I don't 0:07:46.200 --> 0:07:50.640 really like using lms for most product work, even though 0:07:50.680 --> 0:07:52.360 they're good at you know, say like oh add a 0:07:52.360 --> 0:07:54.760 button here that does this. They can be good at that, 0:07:54.880 --> 0:07:58.200 especially if you have a codebase that that works really 0:07:58.280 --> 0:08:02.920 well with lllms. So some programming languages, some tools, some 0:08:03.200 --> 0:08:05.640 we call them libraries and coding you know, they're basically 0:08:05.680 --> 0:08:09.000 like shared software. Some of those things work really well 0:08:09.040 --> 0:08:12.040 with lms because the lms are just trained on the Internet. 0:08:13.120 --> 0:08:16.280 So sites like stack overflow and sites like leak code 0:08:16.320 --> 0:08:18.680 and stuff like that, and so if they have a 0:08:18.760 --> 0:08:22.440 large body of this code to work within their training data, 0:08:22.520 --> 0:08:25.440 like these tools and these languages, they tend to be 0:08:25.440 --> 0:08:28.680 better at writing them. And so you know, I work 0:08:28.920 --> 0:08:32.400 primarily in JavaScript, as do most web application engineers, and 0:08:32.880 --> 0:08:35.640 lms are quite good at JavaScript, but I still don't 0:08:35.720 --> 0:08:38.000 like to use that much because it tends to just 0:08:38.120 --> 0:08:41.760 not understand context very well. 0:08:43.040 --> 0:08:44.800 And mean impracticality. 0:08:45.520 --> 0:08:47.800 So it's kind of just like you know, understanding the 0:08:48.280 --> 0:08:50.640 existing resources that you've already built in your code base. 0:08:50.720 --> 0:08:52.840 So this is like the avoiding writing the same like 0:08:52.920 --> 0:08:56.640 sales tax thing twice. They tend to default to rewriting 0:08:56.720 --> 0:09:03.640 things and they tend to struggle to reuse the same 0:09:03.960 --> 0:09:07.319 like style. So you know, an important thing in coding 0:09:07.559 --> 0:09:11.640 is maintaining consistent like styles of your code. So like 0:09:11.679 --> 0:09:16.600 there's there's a whole theory and practice of actual like 0:09:16.720 --> 0:09:20.960 how your code should look like as like how the texture, 0:09:21.040 --> 0:09:23.560 like how you should avoid, what things you should avoid, 0:09:23.600 --> 0:09:27.160 because there's there's bad patterns, there's good patterns, and you know, 0:09:27.160 --> 0:09:31.040 there's everything in between. I think it's very similar to 0:09:31.160 --> 0:09:35.880 like you know, when image generation AI kind of came 0:09:35.920 --> 0:09:38.040 onto the scene, you had a lot of people being like, 0:09:38.040 --> 0:09:40.240 all right, well, graphic designers are done. You know, they're 0:09:40.240 --> 0:09:42.000 out of a job. They don't need to do anything 0:09:42.200 --> 0:09:44.040 because I can generate a logo now and like it 0:09:44.040 --> 0:09:46.079 looks pretty good. Like I can generate a logo for 0:09:46.160 --> 0:09:48.320 my nail slam or whatever, and look at it looks great. 0:09:48.760 --> 0:09:51.200 But as soon as you try to generate a second 0:09:51.640 --> 0:09:54.120 logo that looks like the first one and you know, 0:09:54.160 --> 0:09:56.840 maybe has a slight modification or or something like that, 0:09:57.240 --> 0:09:59.760 it's really it's really bad at it. It's really bad 0:09:59.760 --> 0:10:01.679 at you using the context and you're like, okay, now 0:10:01.679 --> 0:10:03.640 I need a graphic designer. And as soon as you 0:10:03.679 --> 0:10:09.760 need you know one anyway, yeah, exactly, and so and 0:10:09.960 --> 0:10:12.760 and you know that's it's very similar to engineers, you know, 0:10:12.760 --> 0:10:15.280 they a lot of what you're doing is sort of 0:10:15.320 --> 0:10:20.480 working around context and avoiding redoing things and avoiding moving 0:10:20.559 --> 0:10:22.719 away from your existing styles. 0:10:23.160 --> 0:10:26.400 And what is the critical nature of styles? 0:10:26.480 --> 0:10:26.760 Is that? 0:10:26.800 --> 0:10:30.800 Because is that so the people looking at your staff 0:10:30.920 --> 0:10:33.280 in the future can say, Okay, this is what they 0:10:33.320 --> 0:10:36.400 were going for. This is so that everything doesn't break. 0:10:36.800 --> 0:10:40.840 Yes, exactly. It's about consistency. It's about so there are 0:10:40.880 --> 0:10:43.520 certain you know, for example, JavaScript, which is the main 0:10:43.600 --> 0:10:49.920 language I work in, is almost infamous for basically allowing 0:10:49.960 --> 0:10:53.600 you as the developer to do all sorts of buck 0:10:53.679 --> 0:10:55.920 wild stuff that you should like never do coding wise, 0:10:55.960 --> 0:10:59.480 like a lot of old patterns, a lot of recycled stuff, 0:10:59.559 --> 0:11:01.760 a lot of it just it just lets you. It's 0:11:01.880 --> 0:11:05.920 it's an extremely varied language in the things it supports, 0:11:05.960 --> 0:11:07.480 and so what you want to do when you're writing 0:11:07.480 --> 0:11:10.880 in JavaScript is only use the good parts. You want 0:11:10.920 --> 0:11:13.760 to strategically avoid doing a bunch of bad things. And 0:11:13.840 --> 0:11:16.760 some of those bad things, there's tools that will automatically 0:11:16.800 --> 0:11:29.280 detect if you're doing them. 0:11:29.440 --> 0:11:31.360 What are these bad things? Is it just the job? 0:11:31.679 --> 0:11:34.280 What's so special about Java that it allows you to 0:11:34.640 --> 0:11:34.960 do that? 0:11:35.640 --> 0:11:40.880 Yeah, JavaScript, which is technically different from Java, Yeah, no problem. Basically, 0:11:40.960 --> 0:11:43.320 the reason JavaScript has all these bad parts is because 0:11:43.360 --> 0:11:47.240 it was created in ten days as a toy language 0:11:47.400 --> 0:11:50.640 at Mosaic, which was the precursor to Firefox, you know, 0:11:50.679 --> 0:11:55.200 the first browser that really green traction and sat the Internet. 0:11:55.360 --> 0:11:57.440 I'm old enough to have used Mosaic somehow. 0:11:57.840 --> 0:12:01.240 Yeah. So JavaScript was created in like ten days almost 0:12:01.400 --> 0:12:03.680 like just as like a thing to test, like just 0:12:03.679 --> 0:12:05.760 something to try like, okay, what if we put a 0:12:05.800 --> 0:12:09.160 coding language into the into the browser that people could 0:12:09.200 --> 0:12:14.640 just like ship with their websites. And in response to that, 0:12:14.840 --> 0:12:18.559 other company said, okay, well we need to support JavaScript 0:12:18.559 --> 0:12:20.760 as well, and so when Internet Explorer came out, they 0:12:20.800 --> 0:12:23.439 shipped their own version of JavaScript, which was slightly different. 0:12:24.000 --> 0:12:26.640 And so you have this engineer that was kind of 0:12:26.640 --> 0:12:30.160 built on like these like very hacked to get their principles, 0:12:30.440 --> 0:12:33.920 that was then rebuilt with slightly different principles, and then 0:12:34.200 --> 0:12:36.920 what you have now is thirty years later, sort of 0:12:37.480 --> 0:12:41.280 the conglomeration of all those things into one language, and 0:12:41.880 --> 0:12:45.480 it's improved dramatically in that time. But the nature of 0:12:45.520 --> 0:12:48.240 coding languages is there's a lot of backwards compatibility. Like 0:12:48.240 --> 0:12:51.000 a video game where you know, you you know, you 0:12:51.040 --> 0:12:53.080 want to be able to build a computer that can 0:12:53.160 --> 0:12:56.040 run a game from nineteen ninety one, so you want 0:12:56.040 --> 0:12:58.079 the browser to be able to still run code largely 0:12:58.120 --> 0:13:00.880 that was written in nineteen ninety five or whatever. So 0:13:01.120 --> 0:13:03.560 a lot of that stuff still exists, and what they've 0:13:03.559 --> 0:13:06.680 done is introduce new patterns that can be better, and 0:13:06.720 --> 0:13:09.440 so there's a sort of an equal amount of like 0:13:09.520 --> 0:13:13.800 avoiding that and then also just consistency. So there are 0:13:14.120 --> 0:13:17.640 times where there's two good ways to write ways to 0:13:17.720 --> 0:13:20.120 do something, but you always want to do it only 0:13:20.160 --> 0:13:22.600 in one way so that anytime someone reads it, they 0:13:22.640 --> 0:13:24.360 see that pattern and they say, okay, I know exactly 0:13:24.440 --> 0:13:25.240 what this does. 0:13:25.600 --> 0:13:29.920 Right, And this fundamentally feels like something large of language 0:13:29.920 --> 0:13:32.960 world is bad at because they don't know anything and 0:13:32.960 --> 0:13:35.480 they don't create anything unique either. 0:13:35.520 --> 0:13:37.720 They just repeat yeah exactly. 0:13:37.760 --> 0:13:42.400 I mean, they are statistical inference models, and so they 0:13:42.480 --> 0:13:46.200 are very good at generating what they think should come 0:13:46.240 --> 0:13:51.320 next based on probabilities. And you can when you're training 0:13:51.480 --> 0:13:53.240 a large language model, you can try to push it 0:13:53.240 --> 0:13:55.000 in one way, push it in another way to say 0:13:55.000 --> 0:13:56.920 like no, don't do that, do that. But like trying 0:13:56.960 --> 0:13:59.480 to do that on the broad spectrum of all things 0:13:59.559 --> 0:14:02.640 code is impossible, and so you're going to get things 0:14:02.679 --> 0:14:05.320 that default to the way that they were done on 0:14:05.360 --> 0:14:08.000 the Internet, and so much like so, like like I said, 0:14:08.080 --> 0:14:12.560 JavaScript it is thirty years old, it's extremely it's gone 0:14:12.600 --> 0:14:15.160 through a lot of turk times in terms of patterns 0:14:15.480 --> 0:14:19.280 and so on the Internet, there are just fast swaths 0:14:19.320 --> 0:14:21.720 of terrible JavaScript and you know. 0:14:22.000 --> 0:14:24.320 So it's just these models are trained on bad code 0:14:24.320 --> 0:14:25.480 as well as good code. 0:14:25.840 --> 0:14:29.560 Yeah, exactly, and and that you know, I'm sure they're 0:14:29.560 --> 0:14:33.160 trying very hard when they're training the models to filter 0:14:33.240 --> 0:14:34.600 some of this stuff out, but like trying to do 0:14:34.640 --> 0:14:39.080 a broad filter on hundreds of thousands, millions even of 0:14:39.200 --> 0:14:43.320 what does say again, like decades, Like I mean, JavaScript 0:14:43.360 --> 0:14:45.680 has been around for thirty years. I just have most 0:14:45.960 --> 0:14:48.320 programming languages that are in use right now. 0:14:48.440 --> 0:14:52.920 Right, but this particular one seems particularly chaotic in how 0:14:53.000 --> 0:14:53.600 it's sprawled. 0:14:54.160 --> 0:14:57.600 Yes, JavaScript is a particularly thirty language. Yeah. 0:14:57.720 --> 0:15:00.600 Why So what you're describing, I'm not trying to put 0:15:00.600 --> 0:15:03.040 words in your mouth, is that this stuff doesn't do 0:15:03.160 --> 0:15:05.280 the stuff that everyone's saying it does. Like it's not 0:15:05.400 --> 0:15:09.720 replacing engineers. It doesn't even seem like it could replace engineers. 0:15:10.120 --> 0:15:12.640 Like it's just not it doesn't do In fact, maybe 0:15:12.680 --> 0:15:15.520 it's fair to say that. So it doesn't do software engineering. 0:15:16.400 --> 0:15:19.240 That's almost the perfect way to put it. It does coding, 0:15:19.480 --> 0:15:22.720 but it doesn't do software engineering, and software engineering is 0:15:22.920 --> 0:15:25.680 kind of this broader practice of like everything that comes 0:15:25.720 --> 0:15:30.800 together around coding. So you know, some people really integrate 0:15:31.040 --> 0:15:35.440 lms deeply into their everyday work and they do similar 0:15:35.480 --> 0:15:38.000 work to what I do. So you know, there are 0:15:38.040 --> 0:15:42.760 people who who are primarily having like lms write their code, 0:15:42.800 --> 0:15:48.880 and they can be good engineers, but they're intervening pretty 0:15:48.880 --> 0:15:52.000 constantly from what I understand, and you know, they're having 0:15:52.040 --> 0:15:54.400 to sort of redirect it, make sure it stays on 0:15:54.440 --> 0:15:58.240 patterns and all that stuff, and there's just kind of 0:15:58.280 --> 0:15:59.800 you know, this is the reason that I don't really 0:16:00.360 --> 0:16:03.880 elms that much, is there's just a constant tension between 0:16:04.400 --> 0:16:09.160 the the lack of context they have and you know, 0:16:09.200 --> 0:16:11.360 what you want them to do and constantly reviewing the 0:16:11.400 --> 0:16:13.840 code and making sure it's up to standards when I 0:16:13.880 --> 0:16:15.560 know I could just write it and you know it'll 0:16:15.560 --> 0:16:17.720 take up out as much time then as it would 0:16:17.720 --> 0:16:21.400 take prompting and re prompting. It's just I get better 0:16:21.440 --> 0:16:23.520 code that way, and that's that's what I care about most. 0:16:23.640 --> 0:16:26.160 So do you think it's kind of a mirage al 0:16:26.200 --> 0:16:28.040 most the productivity benefits? 0:16:28.760 --> 0:16:32.320 I think it would vary a lot. I think yes, broadly, 0:16:32.720 --> 0:16:35.240 I think that there are productivity benefits, but like these 0:16:35.320 --> 0:16:38.600 like huge outsides, like oh my gosh, I'm galaxy brained 0:16:38.640 --> 0:16:41.800 right now, I'm doing so much. I think that that 0:16:42.320 --> 0:16:46.840 is largely a mirage based on like extrapolation of like 0:16:47.000 --> 0:16:50.440 small wins. I talk about this a little bit in 0:16:50.440 --> 0:16:53.440 my article that you know, there are times. One thing 0:16:53.480 --> 0:16:56.560 that I use LMS for a lot and not a 0:16:56.600 --> 0:16:59.920 lot sometimes, but that they are really good at is 0:17:00.160 --> 0:17:02.640 you know, sometimes you're writing code and you're like, I 0:17:02.680 --> 0:17:04.520 need to write a thing that I will only run 0:17:04.560 --> 0:17:06.800 once and I will throw in the trash, or I 0:17:06.800 --> 0:17:09.960 need to write a thing and it uses this tool 0:17:10.200 --> 0:17:13.199 that like I don't have the time to learn, but 0:17:13.280 --> 0:17:15.280 like I'd really like to have this tool, Like I'd 0:17:15.320 --> 0:17:19.159 really like to just like use this once and so 0:17:19.320 --> 0:17:21.320 you can you can vibe code something and not really 0:17:21.440 --> 0:17:23.400 understand what the code does. You can run it once 0:17:23.520 --> 0:17:26.000 and not you know, validate the output and make sure 0:17:26.040 --> 0:17:29.680 it works fine, and you know I've saved in that time. 0:17:29.800 --> 0:17:32.399 You know, it might have taken me five hours to 0:17:32.480 --> 0:17:34.679 like learn how to use this tool properly and like 0:17:34.760 --> 0:17:37.160 learn how to use it with good standards. It could 0:17:37.320 --> 0:17:39.560 even have taken me more, and instead I spent twenty minutes. 0:17:40.240 --> 0:17:42.480 You know, wouldn't that be dangerous because you don't know 0:17:42.520 --> 0:17:45.560 how it works exactly. 0:17:45.400 --> 0:17:47.160 Exactly why I like to only use it for these 0:17:47.160 --> 0:17:51.480 like one time like low. You know. So for example, 0:17:51.600 --> 0:17:54.240 I wrote some code the other week and I was 0:17:54.280 --> 0:17:58.320 basically I basically refactored some existing code. So I adjusted 0:17:58.320 --> 0:18:00.720 how it worked a little bit, and I realized there 0:18:00.760 --> 0:18:04.800 was a way that I could break some existing code 0:18:05.080 --> 0:18:07.240 with it. But I didn't have an easy way of 0:18:07.320 --> 0:18:10.520 checking across the entire code based. So we have tests 0:18:10.560 --> 0:18:12.919 in our code base that would you know, catch issues, 0:18:12.920 --> 0:18:15.359 but there's always some code that isn't quite up to 0:18:15.640 --> 0:18:18.480 up to par with tests. And so I had this 0:18:18.560 --> 0:18:21.360 idea that you know, a simple a really simple language 0:18:21.359 --> 0:18:24.320 parser could go through and make sure that this code 0:18:24.359 --> 0:18:26.840 was right, you know, something that is only like thirty 0:18:26.880 --> 0:18:30.639 lines of code. But language parsing is really complicated, and 0:18:30.800 --> 0:18:33.560 the tools to do it are you know there, there 0:18:33.600 --> 0:18:36.320 are a lot of them, and they're very well supported. 0:18:36.359 --> 0:18:40.320 And because language parsing is a huge deal, but it 0:18:40.359 --> 0:18:41.960 would it would take me a lot of time to 0:18:42.040 --> 0:18:43.680 learn that. So I just I just vibe coded a 0:18:43.720 --> 0:18:46.680 little tool. I said, hey, find me every case where 0:18:46.680 --> 0:18:48.680 I use this function, and I call it like this 0:18:49.680 --> 0:18:52.480 in this entire code base, or actually said, I said, 0:18:52.480 --> 0:18:55.199 write a script that will find me that and I 0:18:55.240 --> 0:18:56.800 looked at the code, I was like, yeah, that looks right, 0:18:56.840 --> 0:18:59.720 that looks right, looks right. I ran it, it said 0:18:59.720 --> 0:19:01.639 there was there's no issues in my code base. So 0:19:01.680 --> 0:19:04.160 I intentionally created an issue just to make sure it worked. 0:19:04.400 --> 0:19:07.000 And it worked, it showed it, and so then I 0:19:07.040 --> 0:19:08.640 got rid of the intentional issue. I was like, Okay, 0:19:08.680 --> 0:19:10.440 this is probably good, and I pushed my code and 0:19:10.480 --> 0:19:11.320 it turned out to be good. 0:19:11.480 --> 0:19:14.919 But that's you know, that also seems low stakes. 0:19:15.080 --> 0:19:18.000 Exactly, and it's it's what I would say, wouldn't scale. 0:19:18.040 --> 0:19:21.879 So if I really needed to start doing like language 0:19:21.880 --> 0:19:24.720 person constantly, like I was doing it daily at my job, 0:19:25.080 --> 0:19:28.520 I simply would have to learn. Like you said, you know, 0:19:28.680 --> 0:19:30.520 how do you know that there are any issues? I 0:19:30.520 --> 0:19:33.879 would have to learn the tools, right, So the time 0:19:33.920 --> 0:19:39.440 that I saved was by avoiding learning for this one thing. 0:19:39.560 --> 0:19:42.360 But eventually, like if you're going to make something your 0:19:42.359 --> 0:19:44.119 full time job, you have to learn it because you 0:19:44.160 --> 0:19:46.280 can't fully trust the output. 0:19:46.080 --> 0:19:50.120 Because it isn't your job, like you were just kind 0:19:50.160 --> 0:19:51.240 of mimicking. 0:19:51.720 --> 0:19:55.520 Yeah, and you know you have to. You know, the 0:19:55.760 --> 0:20:01.480 llms make mistakes in occasionally extremely catastrophic ways. There's a 0:20:01.640 --> 0:20:04.160 thing called slop squatting. Have you heard of slap squatting? 0:20:04.440 --> 0:20:06.399 No, but please tell me. I love this term so 0:20:06.520 --> 0:20:07.280 much already. 0:20:07.640 --> 0:20:11.040 Yeah. So basically you might you might have heard like 0:20:11.119 --> 0:20:12.160 domain squatting. 0:20:12.240 --> 0:20:13.960 So this is where like I think I know what 0:20:14.000 --> 0:20:15.840 this is and I'm very excited to hear more. 0:20:16.359 --> 0:20:18.200 Yeah. So, like you know, this was a thing where 0:20:18.240 --> 0:20:21.920 like in nineteen, you know, ninety one, you're like, Okay, 0:20:21.960 --> 0:20:23.840 I think the Internet's gonna be big, so I'm gonna 0:20:23.840 --> 0:20:26.639 grab Nike dot com. Right, I'm just gonna hold it. 0:20:26.680 --> 0:20:30.679 So that's squatting a domain. So slap squatting is basically 0:20:30.800 --> 0:20:36.919 where you the the lms. You know, they're statistical inference machines. 0:20:36.920 --> 0:20:41.200 They don't they don't actually understand uh, what they're doing. 0:20:41.280 --> 0:20:45.520 And so you know, sometimes they will import software. 0:20:45.840 --> 0:20:49.960