Speaker 1: Zone Media. Hello and welcome to this week's Better Offline Monologue.
Speaker 1: I'm your host ed zetron. I heard something really worrying
Speaker 1: the other day about a major hyperscaler. According to the source,
Speaker 1: said hyperscaler was allowing and even encouraging non technical workers
Speaker 1: to deploy code to consumer facing products, specifically those who
Speaker 1: cannot read or write code vibe coding their own projects
Speaker 1: using generative AI, with their code at some point theoretically
Speaker 1: reviewed by an actual software engineer before it gets pushed
Speaker 1: to production. The cold comfort of that review is that
Speaker 1: it assumes that software engineers are at least the ones
Speaker 1: reviewing that code are actually adept at code review, or
Speaker 1: even if they are, that they have sufficient time to
Speaker 1: look over the overlea verbose code that lllms spew. In
Speaker 1: some cases, I've heard management is actively encouraging and even
Speaker 1: mandating these non technical workers to use lms to make
Speaker 1: these features, creating a mutation of tech debt where somebody
Speaker 1: who cannot code uses a machine that doesn't think to
Speaker 1: create code with no intention that nobody really understands, and
Speaker 1: does so at such a velocity that it burdens the
Speaker 1: actual technical workers with constantly having to monitor and fix it.
Speaker 1: LLMS do not understand anything, nor do they think, which
Speaker 1: means any solutions they build or theoretical bug reports that
Speaker 1: they may make are immediately questionable. Their hallucinations are such
Speaker 1: that even features you believe a part of your code,
Speaker 1: after all you can't read it might not be there,
Speaker 1: or might be poorly designed, or might have some sort
Speaker 1: of unforeseen problem that neither you nor the LM are
Speaker 1: aware of, because its training data is based on code
Speaker 1: that already exists, versus any ability to solve novel problems. Also,
Speaker 1: the idea of having any number of non technical peopleship
Speaker 1: code is fucking insane, an indicative of an overwhelming ignorance
Speaker 1: on the part of management. Of even a few years
Speaker 1: of having overwhelming amounts of code written by LMS, even
Speaker 1: by engineers who know what it says and have some
Speaker 1: intention in the prompts they do, is going to create
Speaker 1: a situation where most of the code is written without
Speaker 1: any intention, making it much harder to the debug because
Speaker 1: nobody really knows why it was written that way, because
Speaker 1: LMS doesn't think. I know. I'm repeating myself already, but
Speaker 1: this situation is chilling me, even outside of the vibe coding.
Speaker 1: There's a larger problem of developers writing code with llms
Speaker 1: that they barely review in some cases because they don't
Speaker 1: feel they need to and just skim read it. In
Speaker 1: many many more because their bosses are demanding their ship
Speaker 1: features faster than is responsible or safe. Remember many many
Speaker 1: tech companies are mandating llmuse harassing their workers checking how
Speaker 1: much they use lllms. I've heard this from multiple companies,
Speaker 1: and really it's not just for them, but it's especially
Speaker 1: hard on software engineers. Adding a layer of code written
Speaker 1: by people who quite literally do not understand what it
Speaker 1: says or does guarantees future situations where major services simply break.
Speaker 1: And the more of this nonsensical code that's allowed to
Speaker 1: be stood up on these services, the harder it will
Speaker 1: be to fix. Code isn't just something you write once
Speaker 1: and leave forever. It needs to be maintained by other people.
Speaker 1: Sometimes he is in the future, especially when people keep
Speaker 1: being laid off, which becomes much harder when there's lots
Speaker 1: of code to go through written by an LLM piloted
Speaker 1: by somebody who doesn't know what they're actually doing and
Speaker 1: relies on the LM, which doesn't know anything to tell
Speaker 1: them what's going on. Again, I realize I'm repeating myself,
Speaker 1: But llms do not have thoughts or feelings of knowledge.
Speaker 1: They are generating based on the parameters of their training data,
Speaker 1: which means that all of their code is at best
Speaker 1: an abstraction of somebody else's back and forth with a chatbot.
Speaker 1: The code is not written efficiently or with any consideration
Speaker 1: of who might have to work with it in the future,
Speaker 1: or indeed what other things might be involved. Lms only
Speaker 1: know what they are fed or what they're connected to.
Speaker 1: They don't know the nuances of code. They don't know
Speaker 1: the nuances of software engineering or architecture, which means that
Speaker 1: they only know so much based on their training data
Speaker 1: and the environment around them. Kind of. They don't know
Speaker 1: the nuances of how service let's say Meta or Microsoft's
Speaker 1: has been built over decades, and indeed, the more of
Speaker 1: this code that's used to build those services, the less
Speaker 1: that these companies know about how their actual fucking software works.
Speaker 1: This is setting up the software industry for disaster after disaster,
Speaker 1: and it's already started to happen. To quote the information
Speaker 1: from this week, according to internal Meta communications and an
Speaker 1: incident report seen by the Information A major security alert
Speaker 1: occurred last week after a meta software engineer used an
Speaker 1: in house agent tool similar to openclaw to analyze the
Speaker 1: technical question that another employee had posted on an internal
Speaker 1: discussion forum. After doing the analysis, the AI agent posted
Speaker 1: a response in the discussion forum to the original question,
Speaker 1: offering advice on the technical issue. According to internal communications,
Speaker 1: the agent did so without approval from the employee. It's
Speaker 1: so cool that this is happening. It's so cool, it's great,
Speaker 1: it's actually brilliant. How fucking insan what? And I'm I'm
Speaker 1: kindly going to assume that the person using this knew
Speaker 1: what they were doing. But the idea that we have
Speaker 1: and I think this is what's happening with open source too.
Speaker 1: We have people with llms who are like, yeah, well
Speaker 1: the LM tells me I'm good, so I must be.
Speaker 1: Let me just run this LM pasture problems. It's why
Speaker 1: we're getting all these junk pull requests on GitHub on
Speaker 1: open source projects. People that think they're competent because an
Speaker 1: LM told them to are fucking up the entire software world.
Speaker 1: And according to the Information, Meta systems storing large amounts
Speaker 1: of company and user related data were accessible to engineers
Speaker 1: who didn't have permission to see them, and this was
Speaker 1: marked as second one incident, the second highest level of
Speaker 1: severity on an internal scale that Meta uses to rank
Speaker 1: security incidents. And again that's quoting the information. The incident
Speaker 1: follows multiple problems caused to Amazon by its Kero and qllms.
Speaker 1: I quote Business Inside is Eugene Kim. On March second,
Speaker 1: customers across Amazon marketplaces saw incorrect delivery times when adding
Speaker 1: items to their carts. The incident led to nearly one
Speaker 1: hundred and twenty thousand lost orders and roughly one point
Speaker 1: six million website errors. Amazon's AI tool Q was one
Speaker 1: of the primary contributors that trigger the event, according to
Speaker 1: an internal review. On March fifth, another outage caused a
Speaker 1: ninety nine percent drop in orders across Amazon's North American
Speaker 1: market places, resulting in six point three million lost orders.
Speaker 1: One of the internal documents stated one key factor was
Speaker 1: a production change that was deployed without using a formal
Speaker 1: documentation on an approval process called Model Change Management. Very cool.
Speaker 1: I also want to be clear that it appears that
Speaker 1: these incidents were created by use of these tools. By
Speaker 1: actual software engineers, people that ostensibly know how code and
Speaker 1: software architecture works. Reliance on large language models, especially at
Speaker 1: a time when executives are putting more pressure on engineers
Speaker 1: to deliver more features and ship more code, means that
Speaker 1: software engineers are being incentivized to be sloppy and to
Speaker 1: ship slop itself. There is nothing inherently good about automating code,
Speaker 1: nor is there any inherent value in shipping a lot
Speaker 1: of it fast. Lms convince you that what you're writing
Speaker 1: is good and stable and does the thing you want
Speaker 1: it to. And if your skim reading the outputs, or
Speaker 1: of course unable to read them at all, it's easy
Speaker 1: for you to assume that because you asked the model
Speaker 1: that does not have thoughts whether it thinks you got
Speaker 1: something right, that you actually did so, and that it
Speaker 1: got it right. To be explicit, allowing an LM to
Speaker 1: write all of your code means that you are no
Speaker 1: longer developing code, nor are you learning how to develop code,
Speaker 1: nor are you going to become a better software engineer
Speaker 1: as a result, nor are you solving actual problems. You
Speaker 1: are just handing work over to something and taking dog
Speaker 1: shit out. I'm not saying that all code is using
Speaker 1: lms are inherently bankrupt or anything. But I hear these
Speaker 1: stories about writing all the code, and I'm they give
Speaker 1: me the willies, And I know what I'm saying sounds
Speaker 1: like an insult or hyperbole. I don't mean it in
Speaker 1: that way. If you are just a person looking at code,
Speaker 1: you're only as good as the code the model makes.
Speaker 1: And as Mobitar recently this discussed, these models are built
Speaker 1: to galvanize you, glaze you, and tell you that you
Speaker 1: are remarkable as you barely glance at globs of overwritten
Speaker 1: code that, even if it functions, eventually grows to a
Speaker 1: whole built with no intention or purpose other than what
Speaker 1: the model generated from your prompts. I'm sure there are
Speaker 1: software engineers using these models ethically, who read all the code,
Speaker 1: who have complete industry over it, and use it like
Speaker 1: a glorified auto complain. I can see the value. I'm
Speaker 1: also sure that there are some that are just asking
Speaker 1: it to do stuff, glancing at the code and shipping it.
Speaker 1: It's impossible to measure how many of each camp there are,
Speaker 1: but hearing Spotify's CEO so that its top developers are
Speaker 1: basically not writing code anymore makes me deeply worried, because
Speaker 1: this shit isn't replacing software engineering at all. It's mindlessly
Speaker 1: removing friction and putting the burden of good or right
Speaker 1: on a user that it's intentionally gassing up. And ultimately,
Speaker 1: this entire era is a test of a person's ability
Speaker 1: to understand and appreciate friction. Friction can be a very
Speaker 1: good thing. When I don't understand something, I make an
Speaker 1: effort to do so, and the moment it clicks is magical.
Speaker 1: In the last three years, I've had to teach myself
Speaker 1: a great deal about finance accountancy in the greater technology industry,
Speaker 1: and there have been so many moments where I've walked
Speaker 1: away from the page, frustrated student self doubt that I'd
Speaker 1: never understand something. I eventually did. It took time, It
Speaker 1: really took time, and really, that luxury of time is important,
Speaker 1: and sadly, many software engineers face increasingly deranged deadlines set
Speaker 1: by bosses that don't understand a single fucking thing about
Speaker 1: their job or the software industry itself, let alone what
Speaker 1: lms are capable of, or what responsible software engineering might be.
Speaker 1: The push from above to use these models because they
Speaker 1: can and I quote write code faster than a human
Speaker 1: is a disastrous conflation of fast and good, all because
Speaker 1: of flimsy myths peddled by venture capitalists in the media
Speaker 1: about LM's being able to replace software engineers. It's fucking stupid.
Speaker 1: It's a disgrace, and there are real problems that are
Speaker 1: going to happen as a result. The problem is that
Speaker 1: lms can write all code. Theoretically, they can just put
Speaker 1: the code out that you might have rent yourself. Doesn't
Speaker 1: mean the code is good, or that somebody can read
Speaker 1: it and understand its intention, or that it works, that
Speaker 1: it will work in the future, or that you can
Speaker 1: build any kind of sustainable or I don't know, like
Speaker 1: stable in any way organization on top of it, or
Speaker 1: even that having a lot of code is a good thing,
Speaker 1: both in the present and in the future of any
Speaker 1: company built using this generative code. Adding the variable of
Speaker 1: code written by people who quite literally do not understand
Speaker 1: it guarantee something severe and calamitous in the future, though
Speaker 1: why i'd argue that was the case without their influence.
Speaker 1: Increasing the volume of code contributed to a company naturally
Speaker 1: increases the amount of time needed to read it. And
Speaker 1: the amount of effort needed to maintain it, which naturally
Speaker 1: encourages people to use llms to summarize it. And then well,
Speaker 1: you have to rely on the lms to tell you
Speaker 1: what good looks like, and they don't know a sincle
Speaker 1: fucking thing. And it also creates a new burden on
Speaker 1: the technical workers to have to clean up the slop
Speaker 1: in their day to day lives. Generative code is a
Speaker 1: digital ecological disaster, one that will take years to repair
Speaker 1: thanks to company remits to write as much code as
Speaker 1: fast as possible and use llms as much as possible too.
Speaker 1: Every single person responsible must be held accountable, especially for
Speaker 1: the calamities to come as lazily managed software companies see
Speaker 1: the consequences of building their software on sand I'll see
Speaker 1: you all next week.