1 00:00:02,800 --> 00:00:07,880 Speaker 1: Zone Media. Hello and welcome to this week's Better Offline Monologue. 2 00:00:07,920 --> 00:00:18,800 Speaker 1: I'm your host ed zetron. I heard something really worrying 3 00:00:18,880 --> 00:00:22,160 Speaker 1: the other day about a major hyperscaler. According to the source, 4 00:00:22,160 --> 00:00:26,040 Speaker 1: said hyperscaler was allowing and even encouraging non technical workers 5 00:00:26,040 --> 00:00:30,120 Speaker 1: to deploy code to consumer facing products, specifically those who 6 00:00:30,360 --> 00:00:33,520 Speaker 1: cannot read or write code vibe coding their own projects 7 00:00:33,800 --> 00:00:37,480 Speaker 1: using generative AI, with their code at some point theoretically 8 00:00:37,560 --> 00:00:40,040 Speaker 1: reviewed by an actual software engineer before it gets pushed 9 00:00:40,040 --> 00:00:43,199 Speaker 1: to production. The cold comfort of that review is that 10 00:00:43,280 --> 00:00:45,400 Speaker 1: it assumes that software engineers are at least the ones 11 00:00:45,440 --> 00:00:48,760 Speaker 1: reviewing that code are actually adept at code review, or 12 00:00:48,920 --> 00:00:50,880 Speaker 1: even if they are, that they have sufficient time to 13 00:00:50,920 --> 00:00:54,680 Speaker 1: look over the overlea verbose code that lllms spew. In 14 00:00:54,720 --> 00:00:58,040 Speaker 1: some cases, I've heard management is actively encouraging and even 15 00:00:58,120 --> 00:01:01,319 Speaker 1: mandating these non technical workers to use lms to make 16 00:01:01,400 --> 00:01:04,720 Speaker 1: these features, creating a mutation of tech debt where somebody 17 00:01:04,720 --> 00:01:07,560 Speaker 1: who cannot code uses a machine that doesn't think to 18 00:01:07,600 --> 00:01:11,280 Speaker 1: create code with no intention that nobody really understands, and 19 00:01:11,319 --> 00:01:13,320 Speaker 1: does so at such a velocity that it burdens the 20 00:01:13,360 --> 00:01:16,840 Speaker 1: actual technical workers with constantly having to monitor and fix it. 21 00:01:18,080 --> 00:01:21,640 Speaker 1: LLMS do not understand anything, nor do they think, which 22 00:01:21,680 --> 00:01:24,760 Speaker 1: means any solutions they build or theoretical bug reports that 23 00:01:24,800 --> 00:01:28,560 Speaker 1: they may make are immediately questionable. Their hallucinations are such 24 00:01:28,560 --> 00:01:30,559 Speaker 1: that even features you believe a part of your code, 25 00:01:30,640 --> 00:01:33,000 Speaker 1: after all you can't read it might not be there, 26 00:01:33,200 --> 00:01:35,360 Speaker 1: or might be poorly designed, or might have some sort 27 00:01:35,360 --> 00:01:37,679 Speaker 1: of unforeseen problem that neither you nor the LM are 28 00:01:37,720 --> 00:01:40,200 Speaker 1: aware of, because its training data is based on code 29 00:01:40,240 --> 00:01:44,800 Speaker 1: that already exists, versus any ability to solve novel problems. Also, 30 00:01:44,920 --> 00:01:47,720 Speaker 1: the idea of having any number of non technical peopleship 31 00:01:47,760 --> 00:01:51,320 Speaker 1: code is fucking insane, an indicative of an overwhelming ignorance 32 00:01:51,360 --> 00:01:55,080 Speaker 1: on the part of management. Of even a few years 33 00:01:55,120 --> 00:01:57,800 Speaker 1: of having overwhelming amounts of code written by LMS, even 34 00:01:57,800 --> 00:02:00,160 Speaker 1: by engineers who know what it says and have some 35 00:02:00,840 --> 00:02:03,360 Speaker 1: intention in the prompts they do, is going to create 36 00:02:03,360 --> 00:02:05,800 Speaker 1: a situation where most of the code is written without 37 00:02:05,840 --> 00:02:09,120 Speaker 1: any intention, making it much harder to the debug because 38 00:02:09,160 --> 00:02:11,200 Speaker 1: nobody really knows why it was written that way, because 39 00:02:11,680 --> 00:02:14,960 Speaker 1: LMS doesn't think. I know. I'm repeating myself already, but 40 00:02:15,040 --> 00:02:18,760 Speaker 1: this situation is chilling me, even outside of the vibe coding. 41 00:02:18,800 --> 00:02:21,480 Speaker 1: There's a larger problem of developers writing code with llms 42 00:02:21,480 --> 00:02:24,200 Speaker 1: that they barely review in some cases because they don't 43 00:02:24,200 --> 00:02:26,200 Speaker 1: feel they need to and just skim read it. In 44 00:02:26,280 --> 00:02:28,600 Speaker 1: many many more because their bosses are demanding their ship 45 00:02:28,639 --> 00:02:33,600 Speaker 1: features faster than is responsible or safe. Remember many many 46 00:02:33,680 --> 00:02:38,680 Speaker 1: tech companies are mandating llmuse harassing their workers checking how 47 00:02:38,760 --> 00:02:42,160 Speaker 1: much they use lllms. I've heard this from multiple companies, 48 00:02:42,520 --> 00:02:45,560 Speaker 1: and really it's not just for them, but it's especially 49 00:02:45,560 --> 00:02:49,040 Speaker 1: hard on software engineers. Adding a layer of code written 50 00:02:49,040 --> 00:02:51,600 Speaker 1: by people who quite literally do not understand what it 51 00:02:51,639 --> 00:02:56,239 Speaker 1: says or does guarantees future situations where major services simply break. 52 00:02:56,680 --> 00:02:59,520 Speaker 1: And the more of this nonsensical code that's allowed to 53 00:02:59,560 --> 00:03:01,919 Speaker 1: be stood up on these services, the harder it will 54 00:03:01,919 --> 00:03:05,360 Speaker 1: be to fix. Code isn't just something you write once 55 00:03:05,400 --> 00:03:08,440 Speaker 1: and leave forever. It needs to be maintained by other people. 56 00:03:08,720 --> 00:03:11,000 Speaker 1: Sometimes he is in the future, especially when people keep 57 00:03:11,040 --> 00:03:13,959 Speaker 1: being laid off, which becomes much harder when there's lots 58 00:03:14,000 --> 00:03:16,639 Speaker 1: of code to go through written by an LLM piloted 59 00:03:16,680 --> 00:03:19,280 Speaker 1: by somebody who doesn't know what they're actually doing and 60 00:03:19,320 --> 00:03:22,400 Speaker 1: relies on the LM, which doesn't know anything to tell 61 00:03:22,440 --> 00:03:27,639 Speaker 1: them what's going on. Again, I realize I'm repeating myself, 62 00:03:27,639 --> 00:03:29,919 Speaker 1: But llms do not have thoughts or feelings of knowledge. 63 00:03:29,919 --> 00:03:32,600 Speaker 1: They are generating based on the parameters of their training data, 64 00:03:32,760 --> 00:03:35,160 Speaker 1: which means that all of their code is at best 65 00:03:35,280 --> 00:03:38,480 Speaker 1: an abstraction of somebody else's back and forth with a chatbot. 66 00:03:39,280 --> 00:03:42,000 Speaker 1: The code is not written efficiently or with any consideration 67 00:03:42,040 --> 00:03:43,720 Speaker 1: of who might have to work with it in the future, 68 00:03:43,960 --> 00:03:47,560 Speaker 1: or indeed what other things might be involved. Lms only 69 00:03:47,600 --> 00:03:49,640 Speaker 1: know what they are fed or what they're connected to. 70 00:03:49,920 --> 00:03:52,520 Speaker 1: They don't know the nuances of code. They don't know 71 00:03:52,600 --> 00:03:56,200 Speaker 1: the nuances of software engineering or architecture, which means that 72 00:03:56,240 --> 00:03:59,880 Speaker 1: they only know so much based on their training data 73 00:04:00,040 --> 00:04:02,920 Speaker 1: and the environment around them. Kind of. They don't know 74 00:04:03,200 --> 00:04:07,000 Speaker 1: the nuances of how service let's say Meta or Microsoft's 75 00:04:07,080 --> 00:04:10,200 Speaker 1: has been built over decades, and indeed, the more of 76 00:04:10,200 --> 00:04:13,120 Speaker 1: this code that's used to build those services, the less 77 00:04:13,120 --> 00:04:16,600 Speaker 1: that these companies know about how their actual fucking software works. 78 00:04:17,680 --> 00:04:20,960 Speaker 1: This is setting up the software industry for disaster after disaster, 79 00:04:21,200 --> 00:04:23,960 Speaker 1: and it's already started to happen. To quote the information 80 00:04:24,000 --> 00:04:28,120 Speaker 1: from this week, according to internal Meta communications and an 81 00:04:28,120 --> 00:04:31,479 Speaker 1: incident report seen by the Information A major security alert 82 00:04:31,480 --> 00:04:34,240 Speaker 1: occurred last week after a meta software engineer used an 83 00:04:34,279 --> 00:04:37,359 Speaker 1: in house agent tool similar to openclaw to analyze the 84 00:04:37,360 --> 00:04:40,839 Speaker 1: technical question that another employee had posted on an internal 85 00:04:40,839 --> 00:04:44,640 Speaker 1: discussion forum. After doing the analysis, the AI agent posted 86 00:04:44,680 --> 00:04:47,000 Speaker 1: a response in the discussion forum to the original question, 87 00:04:47,320 --> 00:04:50,920 Speaker 1: offering advice on the technical issue. According to internal communications, 88 00:04:51,320 --> 00:04:54,120 Speaker 1: the agent did so without approval from the employee. It's 89 00:04:54,120 --> 00:04:57,320 Speaker 1: so cool that this is happening. It's so cool, it's great, 90 00:04:57,440 --> 00:05:02,000 Speaker 1: it's actually brilliant. How fucking insan what? And I'm I'm 91 00:05:02,160 --> 00:05:04,599 Speaker 1: kindly going to assume that the person using this knew 92 00:05:04,600 --> 00:05:07,400 Speaker 1: what they were doing. But the idea that we have 93 00:05:07,600 --> 00:05:09,480 Speaker 1: and I think this is what's happening with open source too. 94 00:05:09,680 --> 00:05:11,720 Speaker 1: We have people with llms who are like, yeah, well 95 00:05:11,720 --> 00:05:13,400 Speaker 1: the LM tells me I'm good, so I must be. 96 00:05:13,600 --> 00:05:16,160 Speaker 1: Let me just run this LM pasture problems. It's why 97 00:05:16,160 --> 00:05:19,159 Speaker 1: we're getting all these junk pull requests on GitHub on 98 00:05:19,240 --> 00:05:22,240 Speaker 1: open source projects. People that think they're competent because an 99 00:05:22,320 --> 00:05:26,200 Speaker 1: LM told them to are fucking up the entire software world. 100 00:05:26,400 --> 00:05:29,560 Speaker 1: And according to the Information, Meta systems storing large amounts 101 00:05:29,600 --> 00:05:32,640 Speaker 1: of company and user related data were accessible to engineers 102 00:05:32,680 --> 00:05:34,960 Speaker 1: who didn't have permission to see them, and this was 103 00:05:35,000 --> 00:05:37,800 Speaker 1: marked as second one incident, the second highest level of 104 00:05:37,839 --> 00:05:40,760 Speaker 1: severity on an internal scale that Meta uses to rank 105 00:05:40,800 --> 00:05:44,400 Speaker 1: security incidents. And again that's quoting the information. The incident 106 00:05:44,440 --> 00:05:48,480 Speaker 1: follows multiple problems caused to Amazon by its Kero and qllms. 107 00:05:48,839 --> 00:05:52,839 Speaker 1: I quote Business Inside is Eugene Kim. On March second, 108 00:05:53,160 --> 00:05:57,119 Speaker 1: customers across Amazon marketplaces saw incorrect delivery times when adding 109 00:05:57,160 --> 00:05:59,960 Speaker 1: items to their carts. The incident led to nearly one 110 00:06:00,240 --> 00:06:03,400 Speaker 1: hundred and twenty thousand lost orders and roughly one point 111 00:06:03,440 --> 00:06:08,000 Speaker 1: six million website errors. Amazon's AI tool Q was one 112 00:06:08,040 --> 00:06:10,800 Speaker 1: of the primary contributors that trigger the event, according to 113 00:06:10,839 --> 00:06:14,640 Speaker 1: an internal review. On March fifth, another outage caused a 114 00:06:14,800 --> 00:06:18,400 Speaker 1: ninety nine percent drop in orders across Amazon's North American 115 00:06:18,440 --> 00:06:22,200 Speaker 1: market places, resulting in six point three million lost orders. 116 00:06:22,440 --> 00:06:25,760 Speaker 1: One of the internal documents stated one key factor was 117 00:06:25,800 --> 00:06:28,719 Speaker 1: a production change that was deployed without using a formal 118 00:06:28,760 --> 00:06:34,240 Speaker 1: documentation on an approval process called Model Change Management. Very cool. 119 00:06:35,120 --> 00:06:36,919 Speaker 1: I also want to be clear that it appears that 120 00:06:36,960 --> 00:06:39,200 Speaker 1: these incidents were created by use of these tools. By 121 00:06:39,279 --> 00:06:42,520 Speaker 1: actual software engineers, people that ostensibly know how code and 122 00:06:42,560 --> 00:06:46,680 Speaker 1: software architecture works. Reliance on large language models, especially at 123 00:06:46,680 --> 00:06:49,400 Speaker 1: a time when executives are putting more pressure on engineers 124 00:06:49,400 --> 00:06:51,800 Speaker 1: to deliver more features and ship more code, means that 125 00:06:51,839 --> 00:06:55,080 Speaker 1: software engineers are being incentivized to be sloppy and to 126 00:06:55,120 --> 00:06:59,960 Speaker 1: ship slop itself. There is nothing inherently good about automating code, 127 00:07:00,320 --> 00:07:02,840 Speaker 1: nor is there any inherent value in shipping a lot 128 00:07:02,839 --> 00:07:06,120 Speaker 1: of it fast. Lms convince you that what you're writing 129 00:07:06,160 --> 00:07:08,039 Speaker 1: is good and stable and does the thing you want 130 00:07:08,080 --> 00:07:10,320 Speaker 1: it to. And if your skim reading the outputs, or 131 00:07:10,400 --> 00:07:12,520 Speaker 1: of course unable to read them at all, it's easy 132 00:07:12,560 --> 00:07:14,480 Speaker 1: for you to assume that because you asked the model 133 00:07:14,480 --> 00:07:16,440 Speaker 1: that does not have thoughts whether it thinks you got 134 00:07:16,440 --> 00:07:18,920 Speaker 1: something right, that you actually did so, and that it 135 00:07:19,080 --> 00:07:22,720 Speaker 1: got it right. To be explicit, allowing an LM to 136 00:07:22,760 --> 00:07:24,520 Speaker 1: write all of your code means that you are no 137 00:07:24,600 --> 00:07:27,840 Speaker 1: longer developing code, nor are you learning how to develop code, 138 00:07:28,080 --> 00:07:30,360 Speaker 1: nor are you going to become a better software engineer 139 00:07:30,360 --> 00:07:33,440 Speaker 1: as a result, nor are you solving actual problems. You 140 00:07:33,520 --> 00:07:37,120 Speaker 1: are just handing work over to something and taking dog 141 00:07:37,160 --> 00:07:40,280 Speaker 1: shit out. I'm not saying that all code is using 142 00:07:40,400 --> 00:07:44,280 Speaker 1: lms are inherently bankrupt or anything. But I hear these 143 00:07:44,360 --> 00:07:47,440 Speaker 1: stories about writing all the code, and I'm they give 144 00:07:47,480 --> 00:07:50,760 Speaker 1: me the willies, And I know what I'm saying sounds 145 00:07:50,800 --> 00:07:53,600 Speaker 1: like an insult or hyperbole. I don't mean it in 146 00:07:53,640 --> 00:07:56,240 Speaker 1: that way. If you are just a person looking at code, 147 00:07:56,280 --> 00:07:58,520 Speaker 1: you're only as good as the code the model makes. 148 00:07:58,560 --> 00:08:01,840 Speaker 1: And as Mobitar recently this discussed, these models are built 149 00:08:01,840 --> 00:08:03,880 Speaker 1: to galvanize you, glaze you, and tell you that you 150 00:08:03,920 --> 00:08:07,600 Speaker 1: are remarkable as you barely glance at globs of overwritten 151 00:08:07,640 --> 00:08:10,680 Speaker 1: code that, even if it functions, eventually grows to a 152 00:08:10,720 --> 00:08:13,400 Speaker 1: whole built with no intention or purpose other than what 153 00:08:13,520 --> 00:08:17,480 Speaker 1: the model generated from your prompts. I'm sure there are 154 00:08:17,520 --> 00:08:20,560 Speaker 1: software engineers using these models ethically, who read all the code, 155 00:08:20,560 --> 00:08:22,920 Speaker 1: who have complete industry over it, and use it like 156 00:08:22,960 --> 00:08:26,240 Speaker 1: a glorified auto complain. I can see the value. I'm 157 00:08:26,240 --> 00:08:28,280 Speaker 1: also sure that there are some that are just asking 158 00:08:28,320 --> 00:08:30,800 Speaker 1: it to do stuff, glancing at the code and shipping it. 159 00:08:30,800 --> 00:08:33,520 Speaker 1: It's impossible to measure how many of each camp there are, 160 00:08:33,679 --> 00:08:37,120 Speaker 1: but hearing Spotify's CEO so that its top developers are 161 00:08:37,120 --> 00:08:40,520 Speaker 1: basically not writing code anymore makes me deeply worried, because 162 00:08:40,559 --> 00:08:44,479 Speaker 1: this shit isn't replacing software engineering at all. It's mindlessly 163 00:08:44,559 --> 00:08:47,280 Speaker 1: removing friction and putting the burden of good or right 164 00:08:47,520 --> 00:08:50,760 Speaker 1: on a user that it's intentionally gassing up. And ultimately, 165 00:08:50,840 --> 00:08:53,679 Speaker 1: this entire era is a test of a person's ability 166 00:08:53,679 --> 00:08:57,800 Speaker 1: to understand and appreciate friction. Friction can be a very 167 00:08:57,840 --> 00:09:00,400 Speaker 1: good thing. When I don't understand something, I make an 168 00:09:00,400 --> 00:09:02,679 Speaker 1: effort to do so, and the moment it clicks is magical. 169 00:09:03,120 --> 00:09:05,000 Speaker 1: In the last three years, I've had to teach myself 170 00:09:05,000 --> 00:09:08,240 Speaker 1: a great deal about finance accountancy in the greater technology industry, 171 00:09:08,360 --> 00:09:10,120 Speaker 1: and there have been so many moments where I've walked 172 00:09:10,120 --> 00:09:13,120 Speaker 1: away from the page, frustrated student self doubt that I'd 173 00:09:13,120 --> 00:09:17,800 Speaker 1: never understand something. I eventually did. It took time, It 174 00:09:17,880 --> 00:09:21,199 Speaker 1: really took time, and really, that luxury of time is important, 175 00:09:21,400 --> 00:09:25,040 Speaker 1: and sadly, many software engineers face increasingly deranged deadlines set 176 00:09:25,080 --> 00:09:27,520 Speaker 1: by bosses that don't understand a single fucking thing about 177 00:09:27,520 --> 00:09:30,480 Speaker 1: their job or the software industry itself, let alone what 178 00:09:30,640 --> 00:09:34,640 Speaker 1: lms are capable of, or what responsible software engineering might be. 179 00:09:35,640 --> 00:09:38,079 Speaker 1: The push from above to use these models because they 180 00:09:38,120 --> 00:09:40,360 Speaker 1: can and I quote write code faster than a human 181 00:09:40,720 --> 00:09:44,440 Speaker 1: is a disastrous conflation of fast and good, all because 182 00:09:44,480 --> 00:09:47,200 Speaker 1: of flimsy myths peddled by venture capitalists in the media 183 00:09:47,440 --> 00:09:52,560 Speaker 1: about LM's being able to replace software engineers. It's fucking stupid. 184 00:09:52,840 --> 00:09:56,120 Speaker 1: It's a disgrace, and there are real problems that are 185 00:09:56,160 --> 00:09:59,520 Speaker 1: going to happen as a result. The problem is that 186 00:09:59,679 --> 00:10:04,680 Speaker 1: lms can write all code. Theoretically, they can just put 187 00:10:04,720 --> 00:10:07,040 Speaker 1: the code out that you might have rent yourself. Doesn't 188 00:10:07,040 --> 00:10:09,000 Speaker 1: mean the code is good, or that somebody can read 189 00:10:09,000 --> 00:10:11,760 Speaker 1: it and understand its intention, or that it works, that 190 00:10:11,840 --> 00:10:13,679 Speaker 1: it will work in the future, or that you can 191 00:10:13,679 --> 00:10:17,840 Speaker 1: build any kind of sustainable or I don't know, like 192 00:10:18,360 --> 00:10:21,280 Speaker 1: stable in any way organization on top of it, or 193 00:10:21,320 --> 00:10:22,959 Speaker 1: even that having a lot of code is a good thing, 194 00:10:23,000 --> 00:10:25,160 Speaker 1: both in the present and in the future of any 195 00:10:25,160 --> 00:10:29,040 Speaker 1: company built using this generative code. Adding the variable of 196 00:10:29,120 --> 00:10:32,079 Speaker 1: code written by people who quite literally do not understand 197 00:10:32,120 --> 00:10:36,040 Speaker 1: it guarantee something severe and calamitous in the future, though 198 00:10:36,000 --> 00:10:38,640 Speaker 1: why i'd argue that was the case without their influence. 199 00:10:39,880 --> 00:10:42,880 Speaker 1: Increasing the volume of code contributed to a company naturally 200 00:10:42,880 --> 00:10:46,080 Speaker 1: increases the amount of time needed to read it. And 201 00:10:46,160 --> 00:10:48,840 Speaker 1: the amount of effort needed to maintain it, which naturally 202 00:10:49,000 --> 00:10:54,800 Speaker 1: encourages people to use llms to summarize it. And then well, 203 00:10:54,840 --> 00:10:56,600 Speaker 1: you have to rely on the lms to tell you 204 00:10:56,640 --> 00:10:58,559 Speaker 1: what good looks like, and they don't know a sincle 205 00:10:58,600 --> 00:11:01,800 Speaker 1: fucking thing. And it also creates a new burden on 206 00:11:01,840 --> 00:11:03,880 Speaker 1: the technical workers to have to clean up the slop 207 00:11:04,080 --> 00:11:07,600 Speaker 1: in their day to day lives. Generative code is a 208 00:11:07,640 --> 00:11:11,400 Speaker 1: digital ecological disaster, one that will take years to repair 209 00:11:11,640 --> 00:11:14,160 Speaker 1: thanks to company remits to write as much code as 210 00:11:14,200 --> 00:11:17,520 Speaker 1: fast as possible and use llms as much as possible too. 211 00:11:18,679 --> 00:11:22,840 Speaker 1: Every single person responsible must be held accountable, especially for 212 00:11:22,840 --> 00:11:25,920 Speaker 1: the calamities to come as lazily managed software companies see 213 00:11:25,960 --> 00:11:30,360 Speaker 1: the consequences of building their software on sand I'll see 214 00:11:30,360 --> 00:11:31,199 Speaker 1: you all next week.