Speaker 1: Also media, Hello and welcome to Better Offline. I'm your
Speaker 1: host ed zet Trunk. Also today, I'm joined by Carl Brown,
Speaker 1: a veteran software engineering host of the excellent YouTube channel
Speaker 1: Internet of Bugs. Cayl, thank you for joining me.
Speaker 2: Thanks for having me.
Speaker 1: So I'm going to start with an easy one. What
Speaker 1: is the software developer like? What actually is that?
Speaker 3: So?
Speaker 4: Basically what we do is we take ideas about problems
Speaker 4: that people want to solve, generally, and we write software.
Speaker 4: We write code that tells computers instructions how to make
Speaker 4: the computer do the thing that needs to do to
Speaker 4: solve the problem the person else is to solve.
Speaker 1: Right.
Speaker 4: Gaming programming is a little bit different, but that's most
Speaker 4: software development is basically that.
Speaker 1: And this is another quite silly question, but necessary. How
Speaker 1: much of that is actually writing code?
Speaker 4: It depends on how good you you're the people that
Speaker 4: are asking for stuff. Is As a general rule, I
Speaker 4: would say maybe between ten percent and twenty five percent.
Speaker 1: Okay, just really want to be ten to twenty Even
Speaker 1: if we say thirty percent of the job, which is
Speaker 1: more than you said, that means the majority of this
Speaker 1: job is not actually writing.
Speaker 2: Code right now.
Speaker 4: That's that's largely for folks that are farther up the chain, Right, So,
Speaker 4: if you're fresh out of school and you don't really
Speaker 4: you're not in the job in the you don't understand
Speaker 4: how to manage requirements for any of that kind of stuff.
Speaker 4: Yet someone's going to basically hand you a thing to do,
Speaker 4: and in that kind of case, you're going to be
Speaker 4: spending a lot more time writing code than that. But
Speaker 4: for me, it's you know, it's far far more talking
Speaker 4: to people and stuff than actually code.
Speaker 1: Right. The reason I asked that, and the reason we're
Speaker 1: doing this as well, is the there have been a
Speaker 1: lot of stories around like LM's replacing code as LLLMS
Speaker 1: replacing engineers, claiming that junior software engineers will be a
Speaker 1: thing of the past due to LLLMS. How much validity
Speaker 1: is there in.
Speaker 4: That, Well, when it comes to the really really really
Speaker 4: fresh out of school kids, right.
Speaker 2: That you have to basically break everything.
Speaker 4: Down in hand, the little chunks of work, and LM
Speaker 4: can kind of do that, although the kid will get
Speaker 4: better over time and the LLM is pretty much fixed
Speaker 4: right right, But past that it doesn't do a good
Speaker 4: job of being able to do any kind of long
Speaker 4: term thinking, and that's largely the job, right, I mean
Speaker 4: this is this is not a set of you know,
Speaker 4: I come in today, I do a thing today, I
Speaker 4: come in tomorrow and having no understanding of what happened yesterday,
Speaker 4: and do another self contained thing and so on and
Speaker 4: so forth.
Speaker 2: Right, that's not the job.
Speaker 4: The job is a long sequence of building up on things,
Speaker 4: day after day after day after day until we get
Speaker 4: to the point where the whole thing together works, And
Speaker 4: that's what it's supposed to do.
Speaker 1: So I think that I've known, and one of the
Speaker 1: reasons I had you on as well, is that really
Speaker 1: there are so many of these stories that claiming that,
Speaker 1: like this software engineer's job is gone, that these companies
Speaker 1: be writing all of their code with AI, and it
Speaker 1: doesn't even seem like that is possible. One of your videos,
Speaker 1: you did a really good thing around like the twenty
Speaker 1: to thirty percent a link to this in the notes,
Speaker 1: twenty to thirty percent of code that behind meta and
Speaker 1: I think Google it was is written by AI. Now, again,
Speaker 1: how much valuidity is there to them?
Speaker 4: Well, I mean so if one of the quotes was
Speaker 4: something to the effect of thirty percent of the code
Speaker 4: is suggestions that were given by autocomplete that a human accepted, right,
Speaker 4: which could be as much as you know, the thing said,
Speaker 4: oh wait, you spelled this wrong.
Speaker 2: Let me give you a suggestion about how to spell
Speaker 2: it correctly.
Speaker 4: Right, right, I mean how much of the actual text
Speaker 4: that you write is you know, it is corrected by
Speaker 4: a spell checker?
Speaker 1: Right?
Speaker 4: If all that counts as AI, then what percentage of
Speaker 4: your stuff is written by AI?
Speaker 3: Right?
Speaker 1: Well, in my case, absolutely nothing. But that's just kind
Speaker 1: of a free I'm a just a complete free But no,
Speaker 1: I get your point, and it's without being a code
Speaker 1: of myself. It's something I've really noticed across these stories
Speaker 1: where people just kind of blindly push them out and
Speaker 1: they say, oh, it's twenty to thirty percent of the
Speaker 1: code is written by but there's no verifying this. And
Speaker 1: also it feels like it might create a bigger problem,
Speaker 1: which is say, we accept this idea even though I don't,
Speaker 1: and it sounds like a pretty spurious one kind of
Speaker 1: silly to do so at some point, isn't code not
Speaker 1: just the series of things that you write to make
Speaker 1: a program work. It's connected to a bazillion other things,
Speaker 1: which if you don't know why that was written because
Speaker 1: you had something generated. Is that not a huge problem?
Speaker 2: Yes?
Speaker 4: But worse, what what we're finding when code gets generated
Speaker 4: is that basically you end up doing the same thing
Speaker 4: in a bunch of different places, but in each one
Speaker 4: of those different places.
Speaker 2: You do it a different way.
Speaker 1: Can you give me an example.
Speaker 4: So, for example, when you need to go fetch a
Speaker 4: thing from a server, right, well, you overhear in this
Speaker 4: code you fetch a thing from a server. Over Here
Speaker 4: in the code you fetch a different thing from the server. Normally,
Speaker 4: you'd be able to use the same block of code
Speaker 4: to do that, so that if there's a mistake in it,
Speaker 4: you can change it once and it's fixed everywhere, right, Right,
Speaker 4: But the way the llms work is you say, hey,
Speaker 4: I want to fetch a thing from the server, and
Speaker 4: it says cool, and it writes a whole thing.
Speaker 2: For you that may or may not work the same
Speaker 2: way as the previous one.
Speaker 4: Right, And so now you find, okay, under some circumstances,
Speaker 4: we're having a problem fetching things from the server. I
Speaker 4: don't know which one of these twelve implementations that go
Speaker 4: fetch from the server is the one that's actually causing
Speaker 4: the problem.
Speaker 1: Right, Well, so isn't there isn't there a security issue
Speaker 1: of having large language models, Like wouldn't all the code
Speaker 1: be quite similar or at least more similar depending on
Speaker 1: if everyone's using Claude or everyone's using well get hub copile.
Speaker 2: I guess this Claude now, No, not really.
Speaker 4: It basically kind of picks a random number at the
Speaker 4: beginning and goes okay, So that's the I think if
Speaker 4: it kind of like you deal a deck of cards, right,
Speaker 4: whichever deck of card gets turned over first, that's the
Speaker 4: beginning of the autocomplete that it starts. And so depending
Speaker 4: on which example it's I don't want to say thinking of,
Speaker 4: but depending on which example represents that, I'm grastically over simplifying.
Speaker 4: But depending on which example is represented by that card,
Speaker 4: it's going to go down one path or another.
Speaker 1: Right, And so what they actually what these large language
Speaker 1: model coding tools actually good for? Because I get a
Speaker 1: lot of people who who respond by saying, this is
Speaker 1: proof that AI is a big deal, and I'm just
Speaker 1: kind of like, I'm not even looking for a particular answer,
Speaker 1: just truly what's useful about them?
Speaker 4: So they are decent at when you know what you
Speaker 4: want and what you want is a fairly simple, self
Speaker 4: contained thing, and you know how to tell whether or
Speaker 4: not in the self can think thing does what you want,
Speaker 4: It can type it faster than you can, like wells
Speaker 4: so correct. Basically, Yes, it's like auto complete if you
Speaker 4: if you know exactly what you want. Yeah, I mean
Speaker 4: so I use it a lot because I programm in
Speaker 4: a bunch of different programming languages a lot, right on
Speaker 4: different projects at the same time or on the same
Speaker 4: day or the same week, And it's really easy for
Speaker 4: me to go, Okay, wait, which language am I in? Right?
Speaker 3: Now?
Speaker 2: Okay, how do I do this in this language?
Speaker 1: Right?
Speaker 2: So it's kind of you can.
Speaker 1: Actually understand the generation when it comes.
Speaker 4: To Yeah, it's like I know what kind of loop
Speaker 4: I want, but I don't remember the syntax for this
Speaker 4: particular language where I don't want to. So I use
Speaker 4: it kind of like a Google Translate kind of thing
Speaker 4: to go from one programming language to another sometimes.
Speaker 1: But you wouldn't trust it to build a full software package.
Speaker 2: Oh not at all?
Speaker 1: Why not?
Speaker 2: Well it wouldn't work to start with.
Speaker 1: Why wouldn't it work?
Speaker 4: Well, I mean, so I've done some experimentation on that
Speaker 4: where I've taken fairly complicated.
Speaker 2: Challenges.
Speaker 4: Challenges are intended for programmers to basically get better at
Speaker 4: their craft and that kind of thing. And I've run
Speaker 4: ai you told it, you know, step by step, okay,
Speaker 4: the the challenge says, this is your next step.
Speaker 3: Do this.
Speaker 2: The challenge says, this is your next step.
Speaker 4: Do this on really simple challenges in programming languages like
Speaker 4: Python that it's got a lot and a lot of
Speaker 4: examples for it does okay past the point where you're
Speaker 4: in the really simple kind of language, things they just
Speaker 4: they sometimes get to the point where they can't even
Speaker 4: create anything that builds it all.
Speaker 1: Huh, why is there? Why does so many engineers swear
Speaker 1: by it? Then?
Speaker 4: Honestly, I'm not sure to what extent the engineers are
Speaker 4: swearing by it. I've talked to a lot of folks
Speaker 4: who are like, you know, my group, you know this
Speaker 4: big bank, you know friend of mine, My group is
Speaker 4: getting copilot gemmed on our throats whether we want it
Speaker 4: or not. And the executives are all really excited about it,
Speaker 4: and none of us are interesting.
Speaker 1: So it's executive. I've I've personally had this theory that
Speaker 1: it's like executive pushed, and that it's all about it's
Speaker 1: all about what the bosses want to see rather than
Speaker 1: even do.
Speaker 2: Sorry, there's a lot of wish fulfillment.
Speaker 4: There's a lot of like, we want to not have
Speaker 4: to deal with these programmers anymore, so we would rather
Speaker 4: deal with the AI thing, and we're just gonna hope
Speaker 4: that the AI thing is going to be, you know,
Speaker 4: just as good as the programmers are close to us,
Speaker 4: just as good as the programmers, and not nearly as annoying.
Speaker 1: Seems like a definitional well maybe that's not the right word.
Speaker 1: Seems like the difference between a software engineer and a
Speaker 1: software developer almost because it's not just about flopping code out,
Speaker 1: it's about making sure the code does stuff.
Speaker 2: Yeah, I mean, those terms get mashed to keep yeah,
Speaker 2: I mean.
Speaker 4: So part of the problem is that in like I
Speaker 4: live in Texas, and in Texas you're not allowed to
Speaker 4: call yourself an engineer unless you passed the engineering exam. Right,
Speaker 4: So I can't literally, I literally can't call myself a
Speaker 4: software engineer legally in Texas As. I understand it. I'm
Speaker 4: not a lawyer, but that's my understanding. So it's like
Speaker 4: the terms get all confused.
Speaker 1: Right, So somewhat related, what is it that people misunderstand
Speaker 1: about the job?
Speaker 4: Then, well, I mean, so one of it is what
Speaker 4: you said earlier, which is that a very small percentage
Speaker 4: of the job is actually slinging code. A lot of
Speaker 4: it is basically trying to figure out what it is
Speaker 4: the code should do based on what you've been told
Speaker 4: that the problem, you know, the solution of the problem
Speaker 4: that you're trying to solve.
Speaker 2: Another thing is that.
Speaker 4: A lot of the problem with the job is that
Speaker 4: every little decision builds up over time, and at some
Speaker 4: point a bug is going to happen. They're inevitable, and
Speaker 4: when that happens, basically there's this process where what you
Speaker 4: need to do, if you're being competent, is roll back
Speaker 4: through the series of decisions, figure out what caused that bug,
Speaker 4: and then figure out what other bugs are likely to
Speaker 4: have been caused by that same set of decisions, and
Speaker 4: then fix not just the bug you're working on, but
Speaker 4: the bugs that you know, not just the bug that's
Speaker 4: been reported, but the bugs that might have also been
Speaker 4: caused by the same problem. Right, And that kind of
Speaker 4: long term thinking is not a thing I've ever seen
Speaker 4: LM exhibit at all. I talk about it like l
Speaker 4: limbs or generative AI is good at solving riddles, but
Speaker 4: actual software development is more like solving a murder.
Speaker 1: Yes, you said that in that wonderful video. Yeah, I
Speaker 1: And it almost feels as if we are building towards
Speaker 1: an actual calamity of sorts, maybe not an immediate one.
Speaker 1: Maybe it'll be kind of sectioned off into areas because
Speaker 1: you've got a new generation of young people coming into
Speaker 1: software engineering or what have you, learning to use AI
Speaker 1: tools rather than your videos definitely talk about this as well,
Speaker 1: actually how to develop software and make sure it works,
Speaker 1: and make sure that it has the infrastructural side inline,
Speaker 1: and also that you're building it with the long term
Speaker 1: thinking of someone else might need to understand how this works,
Speaker 1: and they're not learning that, So you've just got a
Speaker 1: generation of kind of pumping the internet and organizations with
Speaker 1: sloppier code.
Speaker 4: Yes, although I mean one of the problems we're having
Speaker 4: at the moment is that the hiring process for really
Speaker 4: junior engineers is actually pretty broken at the moment, and
Speaker 4: a lot of people are not hiring people that are
Speaker 4: fresh out of school because they're expecting that the AI
Speaker 4: will be able to do.
Speaker 2: Basically, a senior or a mid level developer with.
Speaker 4: The benefit of AI, with the benefit of AI that's
Speaker 4: in air quotes, will be able to do the work
Speaker 4: of that person plus a couple of fresh outs that
Speaker 4: they normally would have hired, but they're not hiring at
Speaker 4: the moment. There's some statistics about how the people that
Speaker 4: are fresh out of school these days are historically underemployed
Speaker 4: relative to the general population, at least in the US
Speaker 4: where I live.
Speaker 1: It also feels like there's no intention behind the code,
Speaker 1: like it's just if you're just generating it. You don't
Speaker 1: really know why you made any patity. You could say
Speaker 1: I chose these lines, But is that at some point
Speaker 1: if you have large amounts of software developers using it,
Speaker 1: however large, But the young people in an organization using
Speaker 1: it to generate their code, they're neither learning to write
Speaker 1: better code, nor are they learning how to just learning
Speaker 1: how to fill in blocks they'll kin within the job.
Speaker 2: Yeah, I mean.
Speaker 4: The the trick is that those of us that have
Speaker 4: spent a whole lot of time debugging software right and
Speaker 4: like finding the problems and digging into them and trying
Speaker 4: to figure out what's going on that kind of stuff.
Speaker 4: It's going to be really hard for younger folks to
Speaker 4: get hired into those jobs so that they have time
Speaker 4: to build the experience to be able to do that.
Speaker 4: And I'm afraid we're going to end up with basically
Speaker 4: an older generation or generations retiring and a newer generation
Speaker 4: that hasn't had the experience of doing that kind of debugging.
Speaker 4: And then it's going to be a real mess, especially
Speaker 4: since from what I can tell, the code that the
Speaker 4: AI's generated are a lot buggier and buggier in weirder
Speaker 4: like randomish kind of ways. Stuff just kind of comes
Speaker 4: out of nowhere in a way that I don't. I mean,
Speaker 4: I've debugged code from people that don't speak the same
Speaker 4: languages as I do, you know, all that kind of
Speaker 4: stuff AI code is different. It's just like, Okay, why
Speaker 4: would anyone want to put that block there that doesn't
Speaker 4: have anything to do with what we're trying to do
Speaker 4: at the moment.
Speaker 1: And why is that? Is it just because it's probabilistic.
Speaker 2: I guess so.
Speaker 4: I mean it's hard to say why. I mean, the
Speaker 4: idea of why an LLN does what it does is
Speaker 4: kind of a you know, anybody's guess.
Speaker 1: Yeah, it's just I keep thinking of the word calamity
Speaker 1: because you sent me these studies as well about how
Speaker 1: they found like a downward pressure on the quality of
Speaker 1: code on GitHub. Would you mind walking me through what
Speaker 1: that means?
Speaker 4: Yeah, So basically what that study found, there were there
Speaker 4: have been a couple of them, but what that particular
Speaker 4: study found is that there's what they call code churn
Speaker 4: has gone up. And code churn is basically when you
Speaker 4: push something you like, add a line of code, you
Speaker 4: push it into two test or to production, and then
Speaker 4: in a short period of time, like I don't remember
Speaker 4: exactly what the definition was, like in a month or
Speaker 4: two months, that line of code changes, right. So basically
Speaker 4: what that means is that the line of code that
Speaker 4: got created, somebody decided after it got put in, oh wait, no,
Speaker 4: that doesn't work right, we don't We're not happy with that.
Speaker 4: We're going to change it to be something else, right, right,
Speaker 4: And the percentage of lines or the number of lines
Speaker 4: that that get changed fairly quickly after they get submitted,
Speaker 4: has gone way up since the since the implementation of
Speaker 4: get hub copilot. So and this is across like most
Speaker 4: of the giant you know, millions of lines of codes
Speaker 4: on GitHub and.
Speaker 1: For a simpleton me, why does it being changed? Why
Speaker 1: is changing it so often bad?
Speaker 2: Like? Well, I mean, so, I mean if you do
Speaker 2: it right the first time, you can move on to
Speaker 2: the next thing.
Speaker 3: Ah.
Speaker 1: Right.
Speaker 4: If it's like you know, if you're writing a document
Speaker 4: and you put put the document in there and then
Speaker 4: you like you're get you're in in Google Docs and
Speaker 4: you're like tracking changes and it's like, Okay, this sentence
Speaker 4: has changed seventeen times.
Speaker 2: Obviously the person isn't happy with that, right.
Speaker 1: So the generative code isn't good, right, and so people
Speaker 1: see you need to change.
Speaker 2: That's the presumption, yes, and.
Speaker 1: So I it also said the code quality itself? Is
Speaker 1: that the only way they is that the only way
Speaker 1: they measured it? Or is it there were other things
Speaker 1: as well, So.
Speaker 2: They measured that they measured.
Speaker 1: Uh like moved code.
Speaker 2: Yeah, that the move code.
Speaker 4: The thing I was talking about earlier, where the uh
Speaker 4: you've got a bunch of different places in the code
Speaker 4: that all do the same You try to do the
Speaker 4: same function, but they do it.
Speaker 2: In different ways. Normally, what would happen is you'd have
Speaker 2: your you do it.
Speaker 4: You do a thing here, right, and then at some
Speaker 4: point in the future you need to do that thing
Speaker 4: again in a different place. And so what you do
Speaker 4: is you would move that original block that does the
Speaker 4: thing someplace else, and then you would call that block
Speaker 4: from both places because it already works, right, And then
Speaker 4: that way you've got you know, however, you go that
Speaker 4: stuff from the server, you're fetching it the same way,
Speaker 4: but with this thing. Basically instead of doing that, you've
Speaker 4: got copy paste. Okay, when we put another one here,
Speaker 4: and we put another one here, let me put another
Speaker 4: one here, and it's a it's a maintenance nightmare.
Speaker 1: So for the for the audience as well, how does
Speaker 1: the software developer actually use GitHub? Like really simple stuff?
Speaker 1: I realized, but I think it's important for people to
Speaker 1: It just occurred to be like, this may be something
Speaker 1: that most listeners don't know, which is good to I
Speaker 1: think it's good.
Speaker 2: Yeah.
Speaker 4: So so what we do is we basically make changes
Speaker 4: to code. We get to the point where we the developer,
Speaker 4: are happy with the way it's set up on our machine,
Speaker 4: and then we do what it's called a push, and
Speaker 4: we basically send all that code, submit all that code
Speaker 4: up to get hub, and then theoretically, you know, there
Speaker 4: can be automatic processes that kick in that like check
Speaker 4: that code for particular things and run tests on and
Speaker 4: that kind of stuff. And then at some point we
Speaker 4: have a thing called a poor request, which is basically
Speaker 4: a thing that says, Okay, I would like this to
Speaker 4: go into production now, or more or less, I would
Speaker 4: like this to get promoted into the next phase now,
Speaker 4: and then someone theoretically will look at it and go okay,
Speaker 4: that's fine, and then click the yes button or say, hey,
Speaker 4: you forgot about this, go look at this or that
Speaker 4: kind of thing, right, And the poor requests is kind
Speaker 4: of the unit of work kind of.
Speaker 1: So with get hub you almost use it like an
Speaker 1: organizational code dump, centralize all the code. Sorry just for
Speaker 1: the non coding as well. And I think it's I
Speaker 1: think that the LLLM industry has done a really good
Speaker 1: job of dancing around these terms and selling them to
Speaker 1: people like me. While they weren't selling they didn't work
Speaker 1: on me. I am too stupid, But it's where they've
Speaker 1: just like been like, okay, yeah, well lots of people
Speaker 1: use co pilot, that's good, and this is good because
Speaker 1: software is coding. But it kind of feels like, I
Speaker 1: don't know, all of this is taking the one thing,
Speaker 1: like one major pot out of software development and ruining it.
Speaker 1: And I don't even mean coding. I mean it's the
Speaker 1: intentionality behind software design and infrastructure and maintain Like there's
Speaker 1: it seems like they're removing intention in multiple parts.
Speaker 4: So the way I would say it is when they
Speaker 4: talk about the AI being able to do the work
Speaker 4: of a programmer, what they're doing is they're devaluing all
Speaker 4: of the stuff that's not just hacking code, right, And
Speaker 4: so what they're saying is that basically the job of
Speaker 4: a developer is basically just you know, typing, basically, right,
Speaker 4: And that all of the work that we do to
Speaker 4: understand what the problem actually is and how it needs
Speaker 4: to work, and you know what other problems are likely
Speaker 4: to show up when we try to do that, and
Speaker 4: how to avoid those things as we go and that
Speaker 4: kind of thing, all that work is basically not important.
Speaker 1: And I mean I two words which would probably annoy
Speaker 1: you this is I feel like vibe coating is the
Speaker 1: other part of this. So if I'm correct, correct me
Speaker 1: if I'm wrong, vibe coating is just typing stuff into
Speaker 1: an LEM and software comes out and hopefully it works.
Speaker 2: Yeah. Vibe coating is basically when you intentionally try it.
Speaker 4: Well, I don't know, but intentionally, but basically you make
Speaker 4: a point of not digging into the code and looking
Speaker 4: at what the LM is doing, and you basically say, Okay,
Speaker 4: I would like something that does X right. I would
Speaker 4: like a game where I fly airplanes around a city
Speaker 4: or something right, And then you get what it spits out,
Speaker 4: and then you say, you know, okay, let me try it. Okay, well,
Speaker 4: can we have more airplanes? And okay, can we have
Speaker 4: some balloons with you know, signs on them now? And
Speaker 4: can we do this kind of thing? And then you
Speaker 4: don't think about what the side effects are. You don't
Speaker 4: think about what things could go wrong, You don't think
Speaker 4: about air conditions that kind of stuff, and you just
Speaker 4: hope that this whatever you look at and has the
Speaker 4: right vibe and that you know, if it if it
Speaker 4: looks like kind of what you wanted, that probably it's
Speaker 4: going to be fine, or hopefully it's going to be fine.
Speaker 1: How do you feel about vibe coating?
Speaker 3: So I do it.
Speaker 4: Sometimes vicoding is great for a thing that you're going
Speaker 4: to do once and then throw away.
Speaker 1: Yeah.
Speaker 4: Right, So if it's like, you know, okay, I want
Speaker 4: to I want to do a thing. I want to
Speaker 4: translate this thing to you know, I want to make
Speaker 4: this table go into this format over here or that
Speaker 4: kind of thing. You do it, you get the output
Speaker 4: you want, you throw the code away. No big deal, right,
Speaker 4: like a prototype almost, yeah, basically, And so you know,
Speaker 4: we call them spikes or tracer bullets. Sometimes it's like
Speaker 4: a you know, let me get a thing that works
Speaker 4: at all, right, and then let me see what I
Speaker 4: can learn from that to move into my big maintainable project.
Speaker 4: But for anything that's like, you know, this thing needs
Speaker 4: to run for a while, this thing's needs to not
Speaker 4: get hacked.
Speaker 2: This thing needs to you know, not crash. It's a
Speaker 2: really bad idea.
Speaker 1: Yeah, And at some point I feel like someone building
Speaker 1: a product that they don't really understand the working So
Speaker 1: of it's kind of almost identical to generating a story
Speaker 1: with at GPT, except kind of more complex and more
Speaker 1: prone to errors.
Speaker 3: Yeah.
Speaker 4: And the other thing is that there's an adversarial component, right,
Speaker 4: so people will intentionally try to go hack that thing
Speaker 4: that's sitting on the internet, right, Yeah, in a way
Speaker 4: that they don't intentionally try to go mess with the
Speaker 4: story that you wrote, right, right, And so even if
Speaker 4: it works all by itself, that doesn't mean it's going
Speaker 4: to work when somebody starts pounding on it intentionally trying
Speaker 4: to break it. And if they can break it, then
Speaker 4: that's a whole other set of problems that you now have.
Speaker 1: It feels like quality assurance is just never part. Oh no,
Speaker 1: they are they claiming they're going to do quality assurance
Speaker 1: with large language models. Yet they must some.
Speaker 4: People are Yeah, I mean, to be honest, a lot
Speaker 4: of companies have just been getting rid of quality assurants
Speaker 4: over the years.
Speaker 1: Right.
Speaker 4: Really, when I worked at IBM, we didn't have quality
Speaker 4: assurance at all. They would, no, seriously, they would do this.
Speaker 4: I was in IBM's Cloud group and they would do
Speaker 4: these these what do they call them, uh, packathon kind
Speaker 4: of things that they didn't call that.
Speaker 2: I don't know what they called it.
Speaker 4: But basically everybody in all the other development groups would
Speaker 4: get together and basically bang on the code that was
Speaker 4: about to get released from some other group to try
Speaker 4: to see if they could break it. Right, But they
Speaker 4: didn't have dedicated testers anymore because they decided I guess
Speaker 4: that they didn't think they were worth the money.
Speaker 2: I don't know, but we had some issues because of that.
Speaker 1: When spinal movement happened.
Speaker 4: I was in I don't know, so I was at
Speaker 4: IBM in like twenty seventeen, twenty eighteen, right, so it
Speaker 4: would have been sometime prior to that.
Speaker 2: When I got there, they didn't have any QA folks.
Speaker 1: Really just feels like the it's the management problem as well.
Speaker 1: It's the management cotton people.
Speaker 2: I would think.
Speaker 1: So it's a real shame as well. And I forgive
Speaker 1: me if I'm forgetting exactly where You've mentioned as well
Speaker 1: that there is like compound scar tissue from AI generated code,
Speaker 1: a larger problem of lots of this code being generated
Speaker 1: with AI.
Speaker 2: Well that's that's my expert, right, Yeah, just a potential worry, right.
Speaker 4: Right, so that the more of this we get and
Speaker 4: the more issues that we have, the more stuff we're
Speaker 4: gonna have to dig out of, right, And what I'm
Speaker 4: honestly envisioning at some point in the I don't know
Speaker 4: how long it will take.
Speaker 2: The crypto bubble took way longer to pop than I expected.
Speaker 4: So I don't know how long it's going to be
Speaker 4: before this one does, but I'm expecting that there's going
Speaker 4: to be this big push to try to clean up
Speaker 4: a bunch of this crap here in a few years,
Speaker 4: once people realize that a lot of the code that's
Speaker 4: being written and generated right now is has all of
Speaker 4: these vulnerabilities that nobody's bothering to check for at the moment.
Speaker 1: Right, and those vulnerabilities again non technical way, I read
Speaker 1: that it was like the call upon things on GitHub
Speaker 1: that don't exist, so bad actors create something that resembles
Speaker 1: what it's pulling from.
Speaker 4: That's so that's that's a more specific kind of one.
Speaker 4: I mean, there are a lot of things. I mean,
Speaker 4: so there have been computer viruses since the eighties, right right,
Speaker 4: you know, the Morris worm and that kind of stuff.
Speaker 4: And basically there are no own ways that code if
Speaker 4: you have to write it in a particular way in
Speaker 4: order for it to be secure, right right, And even
Speaker 4: then sometimes people come up with novel ways of making
Speaker 4: something not secure, and.
Speaker 1: How how do you have to write it to make
Speaker 1: it secure? If it's possible to explain.
Speaker 4: Well, I mean, there's a big, long list of rules, right.
Speaker 4: I mean, one thing you can do is you can
Speaker 4: use languages that are what they call safer. But still
Speaker 4: you have to make sure that any input that you
Speaker 4: get from the network, you're really really careful to make
Speaker 4: sure that it doesn't get to overwrite parts of your
Speaker 4: program that actually execute things. You have to make sure
Speaker 4: that it doesn't have the opportunity to be able to
Speaker 4: write to places on your disc that it shouldn't be
Speaker 4: able to write to. You have to be able to
Speaker 4: make sure that it doesn't have access to read data
Speaker 4: that it shouldn't be able to read, you know, all
Speaker 4: that kind of stuff. And when those things don't happen,
Speaker 4: you end up with you know, so and so got hacked.
Speaker 4: You know, turns out that somebody, we think maybe China,
Speaker 4: is reading the email of the you know, people in Congress.
Speaker 4: You get another letter in the mail that says your
Speaker 4: social Security number has been you know, leaked by you know,
Speaker 4: some credit checking firm or something like that.
Speaker 1: Even even like I think it was what the big
Speaker 1: hot target data breach from a while back was through
Speaker 1: the HVAC system. It was it was it's just except
Speaker 1: now we've got and that was with humans writing the
Speaker 1: code right, imagine if we didn't know. Oh god, it
Speaker 1: really does feel like the young people are going like that. Actually, no,
Speaker 1: I take it back. You were talking about agile the
Speaker 1: other day. I'm going to ask you to explain that
Speaker 1: in a second. But it's like, it sounds like for
Speaker 1: almost decades they've been gnawing away at management's been gnawing
Speaker 1: away at the sides of building good software and building
Speaker 1: good software culture.
Speaker 4: Yes, I mean, there's an argument that says we never
Speaker 4: got it right in the first place. But I mean,
Speaker 4: I mean, if you think about it, software has been
Speaker 4: a thing for what fifty years, sixty years, seventy years, right,
Speaker 4: I mean compare that to like construction engineering or bridge
Speaker 4: building or that kind of stuff. Right, we're still, you know,
Speaker 4: relatively speaking, in the infancy, In our infancy as a
Speaker 4: as an industry. You know, it's been a it's been
Speaker 4: a constant evolution, and a lot of times the things
Speaker 4: that were the things that we did to solve a
Speaker 4: problem that we had ended up causing other problems.
Speaker 1: Right.
Speaker 4: So, going back to agile, in the long long ago, right,
Speaker 4: we used to manage software projects the same way we
Speaker 4: manage like build you know, bridge building and building building project,
Speaker 4: you know, construction projects. And it turns out that when
Speaker 4: you're going to build a bridge, you know beforehand what
Speaker 4: you need to build a bridge to do. When you're
Speaker 4: building software, a lot of times people are changing their
Speaker 4: minds as you go. Right, and you build a thing
Speaker 4: and you show it to them and they're like, oh,
Speaker 4: why don't we put this over here, and why don't
Speaker 4: we change this? And that kind of thing right right,
Speaker 4: because you don't have the same kind of constraints physical
Speaker 4: constraints that you do when you're trying to build a bridge.
Speaker 4: And so we gotten this problem where you would create
Speaker 4: these project plans about how you were going to to
Speaker 4: build this thing, and you would never be anywhere close
Speaker 4: to on time because things would change the whole time.
Speaker 4: And so they created this thing called the agile methodology.
Speaker 4: I'm drastically simplifying. There were steps in the middle, but basically,
Speaker 4: so this agile thing is where we instead of saying, okay,
Speaker 4: so this is what the whole project's going to look like,
Speaker 4: we're gonna be doing. We're going to be done in
Speaker 4: six months, and then things changing along the way, we
Speaker 4: basically block off a thing called a sprint. It's a
Speaker 4: week or two or a month, maybe it depends. And
Speaker 4: then you know, everybody picks their own sprint length and
Speaker 4: then you go, okay, I'm only going to talk about
Speaker 4: what's going to happen in the next sprint or two, right,
Speaker 4: And then you get to the end of that two
Speaker 4: weeks and you go, okay, cool, this is what we
Speaker 4: got done. What do we want to do next? And
Speaker 4: then okay, that's what we got done, and what do
Speaker 4: we want to do next? And that kind of thing,
Speaker 4: and that way, as you go, you have the opportunity
Speaker 4: to change things. You have an opportunity to roll changes
Speaker 4: into the process, that kind of thing. Right. The problem
Speaker 4: with that is kind of the same way that dates
Speaker 4: always ran out in water in waterfall, land projects can
Speaker 4: go way way longer than they were expected to at
Speaker 4: the beginning because everybody's focused on just two weeks at
Speaker 4: a time, and you never kind of take a big
Speaker 4: step back like you ought to and go, okay, wait,
Speaker 4: you know we were supposed to be done, you know,
Speaker 4: two months ago.
Speaker 2: When are we going to wrap this up?
Speaker 1: Right? And how has that led to things agting worse?
Speaker 1: Is it that just software culture software development culture has
Speaker 1: been focused on short term perpetually.
Speaker 2: Is the short term is part of it.
Speaker 4: Part of it is there are you know, unscrupulous developers
Speaker 4: out there that basically want to extend the length of
Speaker 4: the project so they can get more money out of.
Speaker 2: It, right right, That's that's always the case. But the
Speaker 2: other thing is that you end up with a real.
Speaker 4: A lot of times, you end up with a real
Speaker 4: lack of like long term planning and long term understanding,
Speaker 4: right right, because everybody's you.
Speaker 2: Know, some kind of thing. You know, companies are only
Speaker 2: worried about what happens next quarter.
Speaker 1: Right.
Speaker 4: If you're only worried about what's going to happen the
Speaker 4: next week or the next four weeks, the things that
Speaker 4: you look on, look at, you know, tends not to
Speaker 4: have the longer term implications that sometimes you need. Right
Speaker 4: And there are times you get close to the end
Speaker 4: and you're like, oh, you know, we didn't think about
Speaker 4: this problem yeaheah.
Speaker 1: And also if you're in a two or three week thing,
Speaker 1: you're probably not thinking even what you did last sprint like.
Speaker 2: It maybe last one, but not like two or three,
Speaker 2: two or three ones ago.
Speaker 1: Is this a problem throughout organizations of all sizes? Is
Speaker 1: this a consultancy problem. Is it everywhere?
Speaker 2: It's most places, Huh, there are, there are some places
Speaker 2: that are.
Speaker 4: Usually in startups, we're a lot more ad hoc and
Speaker 4: we're a lot more you know, focused on trying to
Speaker 4: get things done. The basically, the the the idea is
Speaker 4: the larger you get as an organization, and the more
Speaker 4: money you're throwing at it, and the more management control
Speaker 4: you want, the more of this overhead you put in place,
Speaker 4: and the more complicated things get as just as a
Speaker 4: as a management structure kind of thing.
Speaker 1: And this in the big So this is something you'd
Speaker 1: seem like a Google and an Amazon as well.
Speaker 2: Oh absolutely so.
Speaker 1: Do you do you think it has the same organizational effects.
Speaker 2: Or largely yes.
Speaker 4: The so those organizations tend to be well, those organizations
Speaker 4: historically have tended to be.
Speaker 2: Before the recent like in shitification wave.
Speaker 4: Those I'm assuming I can swear on this, Yeah, yeah,
Speaker 4: those organizations have historically been fairly more engineering driven, which
Speaker 4: means that you typically have people higher in the organization
Speaker 4: that are technical and have been programmers and who understand
Speaker 4: some of the implications, and so they tend to try
Speaker 4: at least we try to run interference with management and
Speaker 4: to try to, you know, make sure everybody's on the
Speaker 4: same page and that kind of stuff. A lot of
Speaker 4: a lot, not all, but a lot of problems can
Speaker 4: get get lessened if you have people in the organization
Speaker 4: that are at higher level whose job is not to
Speaker 4: manage people, but whose job is basically to keep track
Speaker 4: and coordinate between different groups that are doing different technical
Speaker 4: things right, to.
Speaker 1: Make sure people aren't building the same thing I'm guessing,
Speaker 1: or are building the right thing in the right way.
Speaker 4: Yeah, and that how what this group is building is
Speaker 4: going to impact what this group is building at some
Speaker 4: point in the future. And making sure that when you
Speaker 4: get to the point where those two things need to
Speaker 4: talk to each other, they're both aware enough of what
Speaker 4: the other one is doing that the two things hook
Speaker 4: together correctly.
Speaker 1: Yes, So, based on my analysis at these companies, that's
Speaker 1: definitely gone out the window. I mean, even with LLLM integrations.
Speaker 1: So there was a Johnson and Johnson story that went
Speaker 1: out Wall Street General a couple of weeks ago where
Speaker 1: it was like they had eight hundred and ninety LM
Speaker 1: project Generative AI project, of which Taitla the Pereto principle
Speaker 1: wins again ten fifteen percent of them were actually useful.
Speaker 1: And the thing that stunned me about that of them,
Speaker 1: the fact to confirmed my biases, which I love, was
Speaker 1: the fact that they were eight hundred and ninety at
Speaker 1: the fucking things and no one was like should we
Speaker 1: have this many that There was no like like selfare
Speaker 1: engineering culture that was like, hey, are we all chasing
Speaker 1: our tails? Is this useless? But it sounds like they
Speaker 1: were all focused on their little boxes.
Speaker 2: Yeah, I mean so the other thing.
Speaker 4: So understand that again greatly oversimplifying a lot of the
Speaker 4: new stuff that's happened with large language machines, large language models,
Speaker 4: and generative AI. People didn't expect, right. It was kind
Speaker 4: of a surprise when you throw a whole bunch more
Speaker 4: data at a large language model and it started spitting
Speaker 4: out text in a way that nobody really There was
Speaker 4: no like mathematical reason to expect it to be able
Speaker 4: to be as good at generating rottocomplete.
Speaker 2: Stuff as it is.
Speaker 4: It was, right, And so there's this belief that if
Speaker 4: we did the thing and we unexpectedly got more than
Speaker 4: we asked for, if we do more of the thing,
Speaker 4: maybe we'll unexpectedly get more of what we wanted, right
Speaker 4: that hasn't seemed to really pan out the last couple
Speaker 4: of years from what I can see, but that we
Speaker 4: don't really understand enough about this to know whether it's
Speaker 4: going to work, So we might as well throw spaghetti
Speaker 4: at the wall and see if it sticks, because it might.
Speaker 4: Kind of mentality is kind of pervasive at the moment,
Speaker 4: and everybody's there's a lot of fomo. There's a lot
Speaker 4: of like, you know, well, our competitors are probably doing this,
Speaker 4: and so we don't want to get left behind. It
Speaker 4: kind of reminds me of the rumors that they talked
Speaker 4: about back in the eighties when the CIA was doing
Speaker 4: all this psychic research, because supposedly the Russians were doing
Speaker 4: psychic research and it was all complete crap, but both
Speaker 4: sides were convinced that the other side was making some progress,
Speaker 4: and so everybody was dumping a ton of money into it.
Speaker 1: LMM Kultrum exactly. Yes, the title of the episode, So, okay,
Speaker 1: koltr aside, is this something you're seeing in software development though,
Speaker 1: because I know I've seen that in management or it's
Speaker 1: just going to like shut the ship in there. This
Speaker 1: seems like it's an important thing, right or is this
Speaker 1: Are you seeing it within software development?
Speaker 2: So I am seeing it within.
Speaker 4: Software planning, right, So when managers are sitting down and saying, Okay,
Speaker 4: we need to build this new thing, we need to
Speaker 4: create a new group, we need to split this group apart,
Speaker 4: we need to decide what our headcount is going to
Speaker 4: be for next year, there's a lot of okay, and
Speaker 4: what do we think the AI is going to do
Speaker 4: next year?
Speaker 2: And how many headcount do we think that's going to
Speaker 2: save us?
Speaker 4: In that kind of thing, right, There are some companies
Speaker 4: do a Lingo is one, Klarna is one OP sorry, BP,
Speaker 4: the former British Petroleum of what last year had a
Speaker 4: thing where they said they were cutting seventy percent of
Speaker 4: their contract software developers.
Speaker 1: And in most of these they've kind of rolled them
Speaker 1: back as well.
Speaker 2: And I don't think dual lingo has yet.
Speaker 1: This is just being unfit to you. They like a
Speaker 1: day ago, really just like that would kind of It's
Speaker 1: so funny. It's so funny. It's so funny that this
Speaker 1: just keeps happening in exactly the same way. It's like, oh,
Speaker 1: what a surprise, human beings to do stuff. Yeah, but
Speaker 1: it kind of gets back to I think what you've
Speaker 1: said about everything with l lams. It's like you can
Speaker 1: teach something to say, Yeah, I think the right The
Speaker 1: thing you're looking for is this, but you can't teach
Speaker 1: it context. And that's been a point you've made again
Speaker 1: and again, Like it seems the job of a software
Speaker 1: engineer is highly contextual, unless you're like in the earlier days.
Speaker 2: Yeah, and I like it.
Speaker 4: It sometimes to the Memento guy from the Momento movie, right,
Speaker 4: where like can't form long term memories. Then do you
Speaker 4: do you really want the Momento guy to be the
Speaker 4: person that's building the software that makes the seven thirty
Speaker 4: seven max be able to compensate for its control input. Yeah.
Speaker 1: Well, the thing is, though, with that argument, they would argue,
Speaker 1: and I know that there is a better argument here.
Speaker 1: They would argue, well, what if we just give it
Speaker 1: everything that's ever happened? What if we just show every
Speaker 1: single thing we've ever done in GitHub? Surely then it
Speaker 1: would understand.
Speaker 4: So the what I have seen from the papers that
Speaker 4: I have read is that lllms have a basically squishy
Speaker 4: middle context problem kind of the way that you do right. So,
Speaker 4: if somebody gives you a big document to read or
Speaker 4: a big long documentary to watch or something, and then
Speaker 4: they ask you questions. What they're going to find is
Speaker 4: that you remember a lot more from the beginning of
Speaker 4: that and the end of that than you do from
Speaker 4: the middle of that. Right, and lllms have the same
Speaker 4: kind of problem. Right, And the other problem that the
Speaker 4: LMS seem to have is that when you give them
Speaker 4: a whole bunch of instructions, just instructions, polled on instructions,
Speaker 4: pulled on instructions, they can either get confused and forget
Speaker 4: some of the instructions, or they deadlock, or they just
Speaker 4: start going, Okay, I can't satisfy all of these I'm
Speaker 4: not even going to bother to satisfy any of them,
Speaker 4: or they'll pick one or two. The fact that you
Speaker 4: can take a million tokens and you can stick that
Speaker 4: in the memory block that the the GPU is going
Speaker 4: to process, doesn't necessarily believe, doesn't necessarily mean that all
Speaker 4: of the tokens in that memory block are actually going
Speaker 4: to be treated equally and going.
Speaker 2: To be understood. Right in theory, maybe if you could.
Speaker 4: Train your if you could like custom train an LLM
Speaker 4: and modify all of its weights based on exactly what
Speaker 4: your stuff was, and do that like day after day
Speaker 4: after day after day. As things changed, you would theoretically
Speaker 4: get better. I still don't think it would be you know,
Speaker 4: I still think would understand the context as well, but
Speaker 4: that would be ridiculously expensive.
Speaker 1: Yeah, and at that point you could train a person, yes.
Speaker 4: I mean the person would probably be more annoying. So
Speaker 4: that's I mean the point. I mean, a lot of
Speaker 4: this is seems to be really you know, we don't
Speaker 4: like dealing with the Prima Donna programmer kind of thing,
Speaker 4: right that there's this you know, I mean not just programmers, right,
Speaker 4: we don't also don't want to deal with the Prima
Speaker 4: Donna reporters or the Prima Donna illustrators or just want
Speaker 4: to get rid.
Speaker 1: Of these people. Right. He's annoying. They ask for stuff,
Speaker 1: they want money.
Speaker 2: Yeah, and days off and sickly even you know, healthcare, and.
Speaker 1: It's just disgusting, how Dad. It's so it's frustrating as
Speaker 1: well because across software development and everything, but especially with
Speaker 1: self A developers, it feels just very insulting because it
Speaker 1: doesn't seem like this stuff. Actually, here's a better question.
Speaker 1: Have you seen much of an improvement with like one
Speaker 1: to oh three like these reasoning Do you I think
Speaker 1: the reasoning models change things for the beta. If so, how.
Speaker 2: So a little that they don't make as many stupid mistakes.
Speaker 4: It is basically what it what what it boils down
Speaker 4: to going back to your your first thing, though, right,
Speaker 4: I mean so. There was a piece, actually a couple
Speaker 4: of pieces recently. One of them was about, you know,
Speaker 4: tech workers are just like the rest of us, They're miserable.
Speaker 4: There I'll I'll give you blinks to these. The other
Speaker 4: one was a Corey doctor opiece that was like the
Speaker 4: future of Amazon coders is the present of Amazon warehouse
Speaker 4: workers or vice versa. There's there's a lot of there
Speaker 4: has been a lot of deference given to software developers
Speaker 4: over time, because you know, we have been kind of
Speaker 4: the engine that's made a lot of the last twenty
Speaker 4: thirty years work, and there's a desire to make that
Speaker 4: not so anymore, and to make us just as interchangeable
Speaker 4: as everybody else. I guess, you know, from a from
Speaker 4: a economic standpoint, I kind of don't blame them.
Speaker 2: I understand why they're trying to do what they're doing.
Speaker 4: I don't I mean, I don't think that the warehouse
Speaker 4: workers should be treated the way the warehouse workers are treated,
Speaker 4: you know, much less everybody else gets treated that way.
Speaker 4: And it's been a lot worse since the giant layoffs
Speaker 4: at Twitter now X. When that happened and the thing
Speaker 4: didn't crash and completely burn like everybody was or not everybody,
Speaker 4: but a lot of people were expecting it to, the
Speaker 4: the sentiment became, well, maybe all this, all these software
Speaker 4: developers aren't as important as they you know, we've always
Speaker 4: thought they were.
Speaker 2: And you know, we will see over time what the
Speaker 2: end result of that is. My guess is it's going
Speaker 2: to be end up being a mess.
Speaker 4: But you know, I'm a I'm a software developer, right,
Speaker 4: I'm gonna it behooves me for it to be a mess, right,
Speaker 4: So it might just be my bias that's getting in
Speaker 4: the way.
Speaker 1: I actually I think that you're right though, because I
Speaker 1: remember back in twenty twenty one and onward the kind
Speaker 1: of post remote work, the remote works. There was the
Speaker 1: whole anti remote work push, but there was the whole
Speaker 1: quiet quitting and things like that. That's twenty twenty two
Speaker 1: where it's like software engineering, they just they expect to
Speaker 1: be treated so well because twenty twenty one's all the
Speaker 1: insane hiring, right. You saw tech companies like parking software workers.
Speaker 1: I think that played into it as well, where all
Speaker 1: of these companies who chose to pay these software engineers,
Speaker 1: they were the ones that made the offers, got pissed
Speaker 1: off that they'd done. Someone thought we should cut all
Speaker 1: labor down to size, and then along comes coding. Almost
Speaker 1: makes me wonder if most of these venture capitalists talking
Speaker 1: about this don't know how to code themselves. Yeah, gotta wonder.
Speaker 2: I don't know many that do. Yeah, I know some
Speaker 2: that have at some point.
Speaker 1: But the best thing it's at some point it's like
Speaker 1: they're not part of modern software development culture, which I
Speaker 1: know sounds kind of wanky, but I mean, just how
Speaker 1: an organization builds software feels like something they should know.
Speaker 1: But then again, they don't know how to build a
Speaker 1: real organization ethos. Who the fuck? Yeah?
Speaker 2: Well, I mean, honestly a lot of it.
Speaker 4: I've been in organizations that VC's basically killed, right because
Speaker 4: you know, we built a thing. That thing was, you know,
Speaker 4: a reasonable business, But vcs don't want a reasonable business.
Speaker 4: They want either one hundred ex return or they want
Speaker 4: to tax write off, and they don't want anything in between, right, yeah,
Speaker 4: So I mean what what they're looking for is really
Speaker 4: I mean, they're not trying to run a regular business, right,
Speaker 4: They're not trying to do the normal process. They're trying
Speaker 4: to either you know, hit one out of the park
Speaker 4: or throw it away and move on. And so they're
Speaker 4: they're the rules for them are different because what they're
Speaker 4: trying to accomplish is not what the rest of us.
Speaker 2: Are trying to accomplish. As a general rule.
Speaker 1: The theme of the fucking show, it's just like, it's
Speaker 1: just like you have these people that don't code saying
Speaker 1: how cod is should code, like Dario amat Day the
Speaker 1: other day saying that this year we're going to have
Speaker 1: the first one person software company with a billion dollars
Speaker 1: revenue or something like that, and it's just I feel
Speaker 1: like there are some people who should not be allowed
Speaker 1: to speak as much sometimes, but it's just frustrating and insulting.
Speaker 1: And it's but now that you've got me thinking about it,
Speaker 1: it does feel like this is an attempt to reset
Speaker 1: the labor market finally coming for software developers. And I
Speaker 1: don't mean finally in a good way, right.
Speaker 4: I mean, it feels like that being in the being
Speaker 4: in that organ being in that industry at the moment,
Speaker 4: it really feels like that.
Speaker 2: Is it scary right now?
Speaker 1: Is it scary right now?
Speaker 4: Not for me because I'm old enough to be semi retired, right,
Speaker 4: But I mean, I've been talking to a lot of folks.
Speaker 4: I've been having a interviewing how much folks that are
Speaker 4: that are listeners from my channel and kind of trying
Speaker 4: to get a feel for what's going on. And I've
Speaker 4: talked to folks that are you know, like I said that,
Speaker 4: I talked to some folks that were like, you know,
Speaker 4: I work for a big bank. They're cramming copilot, dinner
Speaker 4: throat or eeveryoneted or not. I've talked to some folks
Speaker 4: that are like, every time I sit down with my boss,
Speaker 4: I'm thinking that, you know, this is going to be
Speaker 4: the day that I'm going to find out that my
Speaker 4: group is getting cut the way the other three groups
Speaker 4: in the company is getting cut.
Speaker 2: There's a lot of.
Speaker 4: Artificial productivity requirement increases kind of thing, which is like
Speaker 4: like one, just you know, we you know, we expect
Speaker 4: more tickets closed per you know, two week period than
Speaker 4: you know we've had before because we were giving you
Speaker 4: this AI now, so you ought to be more productive
Speaker 4: that kind of thing.
Speaker 1: Would the ticket necessarily be something that you just write
Speaker 1: toad for it more than just.
Speaker 2: Well, so generally it's more than just that. But but
Speaker 2: generally the ticket.
Speaker 4: That's kind of the way that we track the work
Speaker 4: that we do in a lot of organizations, right, And
Speaker 4: some tickets are like, I'm building a new thing, and
Speaker 4: those are kind of easier to predict. And some tickets
Speaker 4: are this thing isn't behaving right, go figure out where
Speaker 4: the bug is. And those are a lot harder to predict,
Speaker 4: but they have these things. Agile has this thing called
Speaker 4: a velocity graph where basically you see how many tickets
Speaker 4: per person get closed over time, and people want to
Speaker 4: see the slope of that line change because they're giving
Speaker 4: you AI.
Speaker 1: And I'm guessing the people telling you to change that
Speaker 1: don't know what they're talking about.
Speaker 2: That seems to be the case.
Speaker 4: Great, so I mean the good news in theory, right,
Speaker 4: I don't know to what extent this is going to happen,
Speaker 4: but in theory, if they keep telling people, you know,
Speaker 4: that slope of that line should be changing because you
Speaker 4: have AI. Now, over time, if we see the slope
Speaker 4: of that line not changing though, right, then theoretically it
Speaker 4: will be proof that the AI is not providing the
Speaker 4: return that people expect it. Well, you're not using it, right, Well, yes,
Speaker 4: there's always that you're not prompting it, right.
Speaker 1: That is that is basically what I am people. One
Speaker 1: of the many reasons what you want is like, I
Speaker 1: want to have people that actually code on to talk
Speaker 1: about this stuff, because it's really easy as a layman
Speaker 1: myself and for others to just be like, oh but
Speaker 1: this does replace coding, right, and it does? It sounds
Speaker 1: like it really doesn't.
Speaker 2: Like it can help.
Speaker 1: It can be like a force multiplier to an extent,
Speaker 1: but even past the initial steps, it just isn't there.
Speaker 4: Well, I mean, so the best analogy I've always found
Speaker 4: to writing code is actually just writing, right. I Mean,
Speaker 4: you can get chat GBT to spit out a few
Speaker 4: paragraphs for you, right, but you know, you end up with,
Speaker 4: you know, the legal briefs that have the story that's
Speaker 4: made up or the you know, just things that aren't
Speaker 4: connected to reality or stuff that you know, when people
Speaker 4: read them, they're like, I mean, you you can tell
Speaker 4: the difference between AI slot generated you know, like the
Speaker 4: stupid the insert from the Chicago Sun, Yeah, yeah, the
Speaker 4: Philadelphia Inquirer. You know, all the books, all the things
Speaker 4: you can do this summer, right that like made up
Speaker 4: books and all that kind of I mean, like, but
Speaker 4: even even the articles that weren't the ones that we're
Speaker 4: making up stuff. You read the you know, this is
Speaker 4: what's going to be happening this summer. This is what
Speaker 4: the weather's going to be like or whatever. And you're
Speaker 4: reading and you're like this this there's no like insight here,
Speaker 4: there's no thought here, there's no you know, there's nothing
Speaker 4: in here that I get to the end of this.
Speaker 4: I've read the whole thing. I understand the whole thing,
Speaker 4: but I don't have anything I can walk away.
Speaker 1: With, right and I AI agents aren't coming along to
Speaker 1: replace software. But that you're not scared of Devin.
Speaker 4: I am not scared of Devon, so I well, actually
Speaker 4: I kind of am. I am scared that Devon is
Speaker 4: going to make a mess of things and then more
Speaker 4: things are going to get hacked, and that's going to
Speaker 4: end up being worse for everybody.
Speaker 1: On the unit. Right, how would it do that?
Speaker 4: By I mean like we were talking about before, right,
Speaker 4: So when you write code that isn't secure, right, and
Speaker 4: you write code that you know uses a library that's
Speaker 4: got an old version of a thing that they that
Speaker 4: there's a known bug in it, but you don't bother
Speaker 4: to check to see if there's a fix for that bug.
Speaker 4: Or you don't use best practices when it comes to
Speaker 4: writing code and that kind of thing, or you don't
Speaker 4: think about the the kinds of maintainability issues that you're
Speaker 4: going to have, and you do things like you ship
Speaker 4: out code in a in an Internet of things thing
Speaker 4: a light bulb right, or Internet Wi Fi router that
Speaker 4: cannot be patched over the Internet that has a bug
Speaker 4: in it, right, And now it's like that thing is
Speaker 4: going to have a bug in it forever, and you're
Speaker 4: gonna have to find all the ones on the on
Speaker 4: the earth and turn them off before someone's not going
Speaker 4: to be able to take them and be able to
Speaker 4: hack them and use them to attack somebody else from there.
Speaker 1: I mean, IoT is a huge probe low. Oh yeah,
Speaker 1: but the cheap ones have like the spywab stuff and
Speaker 1: panto mining it.
Speaker 4: Just but yeah, the ones, the ones that have they
Speaker 4: have like really nasty vulnerabilities, and they have no way
Speaker 4: of being updated once they leave the factory, right, and
Speaker 4: it's just as long as they're out there, they're going
Speaker 4: to be a problem literally for everybody on the internet.
Speaker 1: Jesus, Well, what can to wrap us up? What can
Speaker 1: a new engineer, someone new to software development? What can
Speaker 1: they learn right now? You've kind of done a video
Speaker 1: on this, but I think it's a good place to
Speaker 1: wrap us up. What can they start learning to actually
Speaker 1: get ahead, to actually prepare for all of this.
Speaker 4: That's a really good question. So you can't these days,
Speaker 4: you can't really be able to be an engineer. You
Speaker 4: can't get hired as an engineer without some ability to
Speaker 4: talk about being able to do prompts and use you know,
Speaker 4: some kind of AI code editor or that kind of thing.
Speaker 4: It's just an expectation of the job. Now, whether it
Speaker 4: should be or not a different thing. The I mean,
Speaker 4: like I said before, there are situations where you tell
Speaker 4: it what you want and it will type faster than
Speaker 4: you possibly can. So you know that's not necessarily bad.
Speaker 4: You need to understand that you need to figure out Well, okay,
Speaker 4: I'll get back to something else. You need to figure
Speaker 4: out basically how to test the thing, right, So how
Speaker 4: do you make sure that the code that it spits
Speaker 4: out does what you meant it to do. And what
Speaker 4: I'm expecting is that we're going to spend more time
Speaker 4: thinking about testing and thinking more about, you know, trying
Speaker 4: to find exceptions and that kind of thing than we
Speaker 4: have in the past, because the code that's actually being
Speaker 4: generated is going to be less likely to be quality
Speaker 4: than it was in the past. Right. The problem is
Speaker 4: it has become the case in the in the programming
Speaker 4: industry that the things you need to do to get
Speaker 4: through the interview to get hired have very little resemblance
Speaker 4: to the things that you actually do on the job
Speaker 4: that you need to actually do a good job. And
Speaker 4: so that's a whole different We could probably have a
Speaker 4: whole other podcast episode just about the interviewing problem.
Speaker 2: But the main thing right now, it's so right now
Speaker 2: the whole hiring thing.
Speaker 4: And this isn't I don't think true for just programmers,
Speaker 4: but it's especially true for programmers. Is all you know,
Speaker 4: bots that customize your resume and write a custom color
Speaker 4: cover letter and then send them over to the submit
Speaker 4: the thing to the bot that's screening the resume and
Speaker 4: screening right, and that getting it to the point where
Speaker 4: you can actually talk to a human is a nightmare
Speaker 4: right now, So the whole hiring system is kind of broken,
Speaker 4: so that the actually getting to the point where you
Speaker 4: can get hired is a nightmare at the moment. But
Speaker 4: the thing that you can do is figure out what
Speaker 4: kinds of things that AI are good at is good at.
Speaker 4: And one of the things that AI is pretty good
Speaker 4: at is things that don't matter as much, right. So,
Speaker 4: like you know, AI can pick the layout of a
Speaker 4: site potentially right, and you could have it picked two
Speaker 4: or three of them, and you can basically do what's
Speaker 4: called an A B test, and you can randomly assign
Speaker 4: people to it. You can figure out which one of
Speaker 4: them performs better, and you can throw the rest.
Speaker 2: Of the money.
Speaker 1: And even then at some point you will probably want
Speaker 1: the design customized.
Speaker 2: Yeah, I mean, but but.
Speaker 4: I think there will be a lot of things where
Speaker 4: people can kind of get something that's kind of good
Speaker 4: enough to get started right, right. And I think that
Speaker 4: to some extent this is going to be kind of
Speaker 4: a boon for the industry in the longer term where
Speaker 4: somebody who can't program right now, but who has some
Speaker 4: idea of kind of what they want can do like
Speaker 4: a vibe coding thing. They can validate that the market
Speaker 4: that they want to try to attack exists, right, and
Speaker 4: that people want to use the kind of thing that
Speaker 4: they built, and then they can bring in somebody to
Speaker 4: actually build it, right, you know what I mean. And
Speaker 4: those kinds of things wouldn't necessarily have been able to
Speaker 4: happen in the complete absence of AI.
Speaker 2: So it's not, I don't think, completely useless.
Speaker 4: And there there's times when as a as a developer,
Speaker 4: there are things that we're not good at, like you know,
Speaker 4: writing marketing copy and that kind of stuff that if
Speaker 4: we're trying to do a project for ourselves, you know,
Speaker 4: a lot of that stuff we can just outsource to
Speaker 4: the AI because it's not the thing that keeps the
Speaker 4: project from actually breaking and getting hacked in that kind
Speaker 4: of thing, right. So it's kind of like there's this
Speaker 4: concept where you need to keep the things that are
Speaker 4: part of your competitive advantage in house, and everything else
Speaker 4: you can kind of outsource to somebody else. The kinds
Speaker 4: of things you can outsource to somebody else are the
Speaker 4: kinds of things that you potentially you could throw an
Speaker 4: AI at because they're.
Speaker 1: Not even even then it's like, it doesn't seem like
Speaker 1: that's a ton of things right now or will.
Speaker 2: Be again, it's the so it's basically two things.
Speaker 4: It's things that where the quality of the thing doesn't
Speaker 4: matter really, right, which every business has those kinds of things, right,
Speaker 4: And they're the kinds of things where you can define
Speaker 4: a metric that you can test the AI against and
Speaker 4: let it try over and over and over and over
Speaker 4: and over again until it gets to the point where
Speaker 4: it's good enough.
Speaker 1: Yeah.
Speaker 4: Right, So if your metric is more people click on
Speaker 4: this button than the button before, right, then you can
Speaker 4: have the AI create a whole bunch of different ways
Speaker 4: to skin that button, right, and then you can say, okay,
Speaker 4: so the one that tested best is the one we're
Speaker 4: going to keep. That's the thing you can throw an
Speaker 4: AI at, right, because you've got a well defined way
Speaker 4: of checking in no telling how long it's going to take,
Speaker 4: but you have a well defined way of checking to
Speaker 4: see if it's working right or not.
Speaker 1: So yeah, I mean, for years I've had the theory
Speaker 1: that this industry was a twenty to twenty five billion
Speaker 1: dollar total addressable market pretending to be a trillion dollar one.
Speaker 1: And everything you're saying really is just it's like you're
Speaker 1: describing things like platform as a service. They like like, yeah,
Speaker 1: things that you use in tandem with very real people
Speaker 1: in intentional ideas.
Speaker 4: Yeah, this is I don't see a world in which
Speaker 4: this is a we replace all the humans. You know
Speaker 4: that the whole Like, you know, this is going to
Speaker 4: displace eighty percent of the white color workers in the world.
Speaker 4: I just you know that the only the only people
Speaker 4: that are really going to be replaced anytime soon are
Speaker 4: people that either weren't doing a great job to start with,
Speaker 4: or people whose bosses don't understand what they were doing
Speaker 4: to the point the boss thought that what they were
Speaker 4: doing mattered. And my guess is that there's going to
Speaker 4: be regret at that point and that at some point
Speaker 4: they're gonna have to bring those people back.
Speaker 1: Well, Carle, this has been such a wonderful conversation. Where
Speaker 1: can people find you?
Speaker 4: I am Internet of Bugs at YouTube is probably the
Speaker 4: easiest place to find me, and then there are links
Speaker 4: on that channel to point at other things.
Speaker 1: And you've been listening to me at Zichron you've been
Speaker 1: listening to Better Offline. Thank you everyone for listening, and yeah,
Speaker 1: we'll catch you next week.
Speaker 2: Thank you for listening to Better Offline.
Speaker 1: The editor and composer of the Better Offline theme song
Speaker 1: is Mattawsowski. You can check out more of his music
Speaker 1: and audio projects at Mattasowski dot com, m A T
Speaker 1: T O S O W s ki dot com. You
Speaker 1: can email me at easy at Better offline dot com
Speaker 1: or visit better Offline dot com to find more podcast
Speaker 1: links and of course, my newsletter. I also really recommend
Speaker 1: you go to chat Where's your Head dot at to
Speaker 1: visit the discord, and go to our slash Better Offline
Speaker 1: to check out I'll Reddit. Thank you so much for listening.
Speaker 3: Better Offline is a production of cool Zone Media. For
Speaker 3: more from cool Zone Media, visit our website Coolzonemedia dot com,
Speaker 3: or check us out on the iHeartRadio app, Apple Podcasts,
Speaker 3: or wherever you get your podcasts