1
00:00:02,400 --> 00:00:08,239
Speaker 1: Also media, Hello and welcome to Better Offline. I'm your

2
00:00:08,280 --> 00:00:22,759
Speaker 1: host ed zet Trunk. Also today, I'm joined by Carl Brown,

3
00:00:22,800 --> 00:00:25,880
Speaker 1: a veteran software engineering host of the excellent YouTube channel

4
00:00:25,920 --> 00:00:28,120
Speaker 1: Internet of Bugs. Cayl, thank you for joining me.

5
00:00:28,400 --> 00:00:29,160
Speaker 2: Thanks for having me.

6
00:00:29,720 --> 00:00:32,199
Speaker 1: So I'm going to start with an easy one. What

7
00:00:32,360 --> 00:00:34,440
Speaker 1: is the software developer like? What actually is that?

8
00:00:35,479 --> 00:00:35,639
Speaker 3: So?

9
00:00:35,920 --> 00:00:41,640
Speaker 4: Basically what we do is we take ideas about problems

10
00:00:41,640 --> 00:00:47,599
Speaker 4: that people want to solve, generally, and we write software.

11
00:00:47,640 --> 00:00:52,920
Speaker 4: We write code that tells computers instructions how to make

12
00:00:53,440 --> 00:00:56,960
Speaker 4: the computer do the thing that needs to do to

13
00:00:57,000 --> 00:00:58,720
Speaker 4: solve the problem the person else is to solve.

14
00:00:59,200 --> 00:00:59,400
Speaker 1: Right.

15
00:01:01,000 --> 00:01:04,760
Speaker 4: Gaming programming is a little bit different, but that's most

16
00:01:04,760 --> 00:01:06,280
Speaker 4: software development is basically that.

17
00:01:06,840 --> 00:01:10,679
Speaker 1: And this is another quite silly question, but necessary. How

18
00:01:10,800 --> 00:01:12,440
Speaker 1: much of that is actually writing code?

19
00:01:14,400 --> 00:01:17,760
Speaker 4: It depends on how good you you're the people that

20
00:01:17,800 --> 00:01:20,760
Speaker 4: are asking for stuff. Is As a general rule, I

21
00:01:20,760 --> 00:01:25,200
Speaker 4: would say maybe between ten percent and twenty five percent.

22
00:01:25,520 --> 00:01:28,080
Speaker 1: Okay, just really want to be ten to twenty Even

23
00:01:28,080 --> 00:01:29,800
Speaker 1: if we say thirty percent of the job, which is

24
00:01:29,840 --> 00:01:32,600
Speaker 1: more than you said, that means the majority of this

25
00:01:32,680 --> 00:01:34,200
Speaker 1: job is not actually writing.

26
00:01:33,920 --> 00:01:35,520
Speaker 2: Code right now.

27
00:01:35,800 --> 00:01:40,720
Speaker 4: That's that's largely for folks that are farther up the chain, Right, So,

28
00:01:40,720 --> 00:01:42,919
Speaker 4: if you're fresh out of school and you don't really

29
00:01:42,959 --> 00:01:45,560
Speaker 4: you're not in the job in the you don't understand

30
00:01:45,560 --> 00:01:47,560
Speaker 4: how to manage requirements for any of that kind of stuff.

31
00:01:47,600 --> 00:01:49,760
Speaker 4: Yet someone's going to basically hand you a thing to do,

32
00:01:50,680 --> 00:01:52,160
Speaker 4: and in that kind of case, you're going to be

33
00:01:52,160 --> 00:01:53,920
Speaker 4: spending a lot more time writing code than that. But

34
00:01:54,040 --> 00:01:58,600
Speaker 4: for me, it's you know, it's far far more talking

35
00:01:58,640 --> 00:02:00,560
Speaker 4: to people and stuff than actually code.

36
00:02:01,080 --> 00:02:03,120
Speaker 1: Right. The reason I asked that, and the reason we're

37
00:02:03,160 --> 00:02:05,160
Speaker 1: doing this as well, is the there have been a

38
00:02:05,160 --> 00:02:08,240
Speaker 1: lot of stories around like LM's replacing code as LLLMS

39
00:02:08,280 --> 00:02:12,880
Speaker 1: replacing engineers, claiming that junior software engineers will be a

40
00:02:12,880 --> 00:02:16,800
Speaker 1: thing of the past due to LLLMS. How much validity

41
00:02:16,880 --> 00:02:17,400
Speaker 1: is there in.

42
00:02:17,320 --> 00:02:20,480
Speaker 4: That, Well, when it comes to the really really really

43
00:02:20,560 --> 00:02:22,560
Speaker 4: fresh out of school kids, right.

44
00:02:23,960 --> 00:02:25,799
Speaker 2: That you have to basically break everything.

45
00:02:25,480 --> 00:02:28,040
Speaker 4: Down in hand, the little chunks of work, and LM

46
00:02:28,080 --> 00:02:31,320
Speaker 4: can kind of do that, although the kid will get

47
00:02:31,360 --> 00:02:34,000
Speaker 4: better over time and the LLM is pretty much fixed

48
00:02:34,200 --> 00:02:39,720
Speaker 4: right right, But past that it doesn't do a good

49
00:02:39,800 --> 00:02:42,280
Speaker 4: job of being able to do any kind of long

50
00:02:42,360 --> 00:02:46,400
Speaker 4: term thinking, and that's largely the job, right, I mean

51
00:02:46,520 --> 00:02:49,200
Speaker 4: this is this is not a set of you know,

52
00:02:50,280 --> 00:02:52,440
Speaker 4: I come in today, I do a thing today, I

53
00:02:52,520 --> 00:02:55,480
Speaker 4: come in tomorrow and having no understanding of what happened yesterday,

54
00:02:55,800 --> 00:02:58,079
Speaker 4: and do another self contained thing and so on and

55
00:02:58,160 --> 00:02:58,520
Speaker 4: so forth.

56
00:02:58,600 --> 00:02:59,480
Speaker 2: Right, that's not the job.

57
00:02:59,520 --> 00:03:03,160
Speaker 4: The job is a long sequence of building up on things,

58
00:03:03,360 --> 00:03:05,200
Speaker 4: day after day after day after day until we get

59
00:03:05,240 --> 00:03:08,240
Speaker 4: to the point where the whole thing together works, And

60
00:03:08,280 --> 00:03:09,200
Speaker 4: that's what it's supposed to do.

61
00:03:09,880 --> 00:03:13,440
Speaker 1: So I think that I've known, and one of the

62
00:03:13,480 --> 00:03:15,480
Speaker 1: reasons I had you on as well, is that really

63
00:03:15,600 --> 00:03:17,600
Speaker 1: there are so many of these stories that claiming that,

64
00:03:17,800 --> 00:03:21,440
Speaker 1: like this software engineer's job is gone, that these companies

65
00:03:21,520 --> 00:03:24,400
Speaker 1: be writing all of their code with AI, and it

66
00:03:24,680 --> 00:03:26,960
Speaker 1: doesn't even seem like that is possible. One of your videos,

67
00:03:27,000 --> 00:03:28,880
Speaker 1: you did a really good thing around like the twenty

68
00:03:28,960 --> 00:03:30,720
Speaker 1: to thirty percent a link to this in the notes,

69
00:03:30,720 --> 00:03:33,799
Speaker 1: twenty to thirty percent of code that behind meta and

70
00:03:33,880 --> 00:03:37,120
Speaker 1: I think Google it was is written by AI. Now, again,

71
00:03:37,160 --> 00:03:38,720
Speaker 1: how much valuidity is there to them?

72
00:03:39,040 --> 00:03:42,600
Speaker 4: Well, I mean so if one of the quotes was

73
00:03:42,680 --> 00:03:45,080
Speaker 4: something to the effect of thirty percent of the code

74
00:03:45,280 --> 00:03:49,040
Speaker 4: is suggestions that were given by autocomplete that a human accepted, right,

75
00:03:49,840 --> 00:03:53,840
Speaker 4: which could be as much as you know, the thing said,

76
00:03:53,960 --> 00:03:55,200
Speaker 4: oh wait, you spelled this wrong.

77
00:03:55,320 --> 00:03:57,000
Speaker 2: Let me give you a suggestion about how to spell

78
00:03:57,040 --> 00:03:57,400
Speaker 2: it correctly.

79
00:03:57,480 --> 00:04:01,160
Speaker 4: Right, right, I mean how much of the actual text

80
00:04:01,240 --> 00:04:04,880
Speaker 4: that you write is you know, it is corrected by

81
00:04:04,920 --> 00:04:05,560
Speaker 4: a spell checker?

82
00:04:05,640 --> 00:04:05,760
Speaker 1: Right?

83
00:04:05,800 --> 00:04:07,880
Speaker 4: If all that counts as AI, then what percentage of

84
00:04:07,880 --> 00:04:08,960
Speaker 4: your stuff is written by AI?

85
00:04:09,080 --> 00:04:09,200
Speaker 3: Right?

86
00:04:09,320 --> 00:04:12,560
Speaker 1: Well, in my case, absolutely nothing. But that's just kind

87
00:04:12,600 --> 00:04:14,560
Speaker 1: of a free I'm a just a complete free But no,

88
00:04:14,680 --> 00:04:17,800
Speaker 1: I get your point, and it's without being a code

89
00:04:17,800 --> 00:04:21,159
Speaker 1: of myself. It's something I've really noticed across these stories

90
00:04:21,200 --> 00:04:23,960
Speaker 1: where people just kind of blindly push them out and

91
00:04:24,040 --> 00:04:25,600
Speaker 1: they say, oh, it's twenty to thirty percent of the

92
00:04:25,640 --> 00:04:29,240
Speaker 1: code is written by but there's no verifying this. And

93
00:04:29,360 --> 00:04:32,679
Speaker 1: also it feels like it might create a bigger problem,

94
00:04:32,720 --> 00:04:36,240
Speaker 1: which is say, we accept this idea even though I don't,

95
00:04:36,400 --> 00:04:38,920
Speaker 1: and it sounds like a pretty spurious one kind of

96
00:04:38,920 --> 00:04:42,720
Speaker 1: silly to do so at some point, isn't code not

97
00:04:42,960 --> 00:04:45,479
Speaker 1: just the series of things that you write to make

98
00:04:45,520 --> 00:04:48,320
Speaker 1: a program work. It's connected to a bazillion other things,

99
00:04:48,400 --> 00:04:51,960
Speaker 1: which if you don't know why that was written because

100
00:04:51,960 --> 00:04:54,840
Speaker 1: you had something generated. Is that not a huge problem?

101
00:04:55,800 --> 00:04:56,040
Speaker 2: Yes?

102
00:04:56,279 --> 00:05:01,440
Speaker 4: But worse, what what we're finding when code gets generated

103
00:05:01,560 --> 00:05:03,800
Speaker 4: is that basically you end up doing the same thing

104
00:05:03,880 --> 00:05:06,280
Speaker 4: in a bunch of different places, but in each one

105
00:05:06,320 --> 00:05:07,640
Speaker 4: of those different places.

106
00:05:07,360 --> 00:05:08,320
Speaker 2: You do it a different way.

107
00:05:08,839 --> 00:05:09,840
Speaker 1: Can you give me an example.

108
00:05:10,520 --> 00:05:14,240
Speaker 4: So, for example, when you need to go fetch a

109
00:05:14,279 --> 00:05:16,600
Speaker 4: thing from a server, right, well, you overhear in this

110
00:05:16,680 --> 00:05:18,280
Speaker 4: code you fetch a thing from a server. Over Here

111
00:05:18,320 --> 00:05:21,000
Speaker 4: in the code you fetch a different thing from the server. Normally,

112
00:05:21,040 --> 00:05:22,920
Speaker 4: you'd be able to use the same block of code

113
00:05:22,920 --> 00:05:24,440
Speaker 4: to do that, so that if there's a mistake in it,

114
00:05:24,520 --> 00:05:26,720
Speaker 4: you can change it once and it's fixed everywhere, right, Right,

115
00:05:26,920 --> 00:05:28,800
Speaker 4: But the way the llms work is you say, hey,

116
00:05:28,839 --> 00:05:29,920
Speaker 4: I want to fetch a thing from the server, and

117
00:05:29,960 --> 00:05:31,480
Speaker 4: it says cool, and it writes a whole thing.

118
00:05:31,440 --> 00:05:32,920
Speaker 2: For you that may or may not work the same

119
00:05:32,920 --> 00:05:33,880
Speaker 2: way as the previous one.

120
00:05:33,960 --> 00:05:38,160
Speaker 4: Right, And so now you find, okay, under some circumstances,

121
00:05:38,240 --> 00:05:40,120
Speaker 4: we're having a problem fetching things from the server. I

122
00:05:40,160 --> 00:05:42,120
Speaker 4: don't know which one of these twelve implementations that go

123
00:05:42,200 --> 00:05:44,040
Speaker 4: fetch from the server is the one that's actually causing

124
00:05:44,080 --> 00:05:44,520
Speaker 4: the problem.

125
00:05:45,000 --> 00:05:48,239
Speaker 1: Right, Well, so isn't there isn't there a security issue

126
00:05:48,279 --> 00:05:53,120
Speaker 1: of having large language models, Like wouldn't all the code

127
00:05:53,160 --> 00:05:55,840
Speaker 1: be quite similar or at least more similar depending on

128
00:05:56,040 --> 00:05:58,920
Speaker 1: if everyone's using Claude or everyone's using well get hub copile.

129
00:05:58,960 --> 00:06:00,839
Speaker 2: I guess this Claude now, No, not really.

130
00:06:01,200 --> 00:06:03,240
Speaker 4: It basically kind of picks a random number at the

131
00:06:03,240 --> 00:06:05,640
Speaker 4: beginning and goes okay, So that's the I think if

132
00:06:05,640 --> 00:06:07,200
Speaker 4: it kind of like you deal a deck of cards, right,

133
00:06:07,279 --> 00:06:09,600
Speaker 4: whichever deck of card gets turned over first, that's the

134
00:06:09,720 --> 00:06:12,640
Speaker 4: beginning of the autocomplete that it starts. And so depending

135
00:06:12,680 --> 00:06:15,360
Speaker 4: on which example it's I don't want to say thinking of,

136
00:06:15,800 --> 00:06:20,240
Speaker 4: but depending on which example represents that, I'm grastically over simplifying.

137
00:06:20,240 --> 00:06:22,320
Speaker 4: But depending on which example is represented by that card,

138
00:06:22,560 --> 00:06:24,200
Speaker 4: it's going to go down one path or another.

139
00:06:24,680 --> 00:06:28,360
Speaker 1: Right, And so what they actually what these large language

140
00:06:28,400 --> 00:06:30,640
Speaker 1: model coding tools actually good for? Because I get a

141
00:06:30,680 --> 00:06:33,400
Speaker 1: lot of people who who respond by saying, this is

142
00:06:33,560 --> 00:06:36,240
Speaker 1: proof that AI is a big deal, and I'm just

143
00:06:36,320 --> 00:06:38,479
Speaker 1: kind of like, I'm not even looking for a particular answer,

144
00:06:38,600 --> 00:06:40,160
Speaker 1: just truly what's useful about them?

145
00:06:40,839 --> 00:06:44,480
Speaker 4: So they are decent at when you know what you

146
00:06:44,680 --> 00:06:48,000
Speaker 4: want and what you want is a fairly simple, self

147
00:06:48,080 --> 00:06:52,160
Speaker 4: contained thing, and you know how to tell whether or

148
00:06:52,160 --> 00:06:53,880
Speaker 4: not in the self can think thing does what you want,

149
00:06:54,680 --> 00:06:57,600
Speaker 4: It can type it faster than you can, like wells

150
00:06:57,600 --> 00:07:01,800
Speaker 4: so correct. Basically, Yes, it's like auto complete if you

151
00:07:01,960 --> 00:07:03,640
Speaker 4: if you know exactly what you want. Yeah, I mean

152
00:07:03,880 --> 00:07:06,280
Speaker 4: so I use it a lot because I programm in

153
00:07:06,320 --> 00:07:09,400
Speaker 4: a bunch of different programming languages a lot, right on

154
00:07:09,440 --> 00:07:12,120
Speaker 4: different projects at the same time or on the same

155
00:07:12,200 --> 00:07:14,360
Speaker 4: day or the same week, And it's really easy for

156
00:07:14,440 --> 00:07:16,280
Speaker 4: me to go, Okay, wait, which language am I in? Right?

157
00:07:16,320 --> 00:07:16,400
Speaker 3: Now?

158
00:07:16,440 --> 00:07:18,160
Speaker 2: Okay, how do I do this in this language?

159
00:07:18,200 --> 00:07:18,320
Speaker 1: Right?

160
00:07:18,400 --> 00:07:19,240
Speaker 2: So it's kind of you can.

161
00:07:19,160 --> 00:07:21,440
Speaker 1: Actually understand the generation when it comes.

162
00:07:21,320 --> 00:07:23,200
Speaker 4: To Yeah, it's like I know what kind of loop

163
00:07:23,240 --> 00:07:24,640
Speaker 4: I want, but I don't remember the syntax for this

164
00:07:24,680 --> 00:07:26,520
Speaker 4: particular language where I don't want to. So I use

165
00:07:26,560 --> 00:07:28,840
Speaker 4: it kind of like a Google Translate kind of thing

166
00:07:28,920 --> 00:07:31,040
Speaker 4: to go from one programming language to another sometimes.

167
00:07:32,160 --> 00:07:36,680
Speaker 1: But you wouldn't trust it to build a full software package.

168
00:07:36,480 --> 00:07:37,080
Speaker 2: Oh not at all?

169
00:07:37,720 --> 00:07:38,040
Speaker 1: Why not?

170
00:07:38,800 --> 00:07:40,080
Speaker 2: Well it wouldn't work to start with.

171
00:07:40,760 --> 00:07:41,600
Speaker 1: Why wouldn't it work?

172
00:07:42,000 --> 00:07:43,840
Speaker 4: Well, I mean, so I've done some experimentation on that

173
00:07:44,080 --> 00:07:48,280
Speaker 4: where I've taken fairly complicated.

174
00:07:50,160 --> 00:07:50,680
Speaker 2: Challenges.

175
00:07:51,000 --> 00:07:54,120
Speaker 4: Challenges are intended for programmers to basically get better at

176
00:07:54,160 --> 00:07:56,280
Speaker 4: their craft and that kind of thing. And I've run

177
00:07:56,440 --> 00:07:58,800
Speaker 4: ai you told it, you know, step by step, okay,

178
00:07:59,640 --> 00:08:01,520
Speaker 4: the the challenge says, this is your next step.

179
00:08:01,560 --> 00:08:01,760
Speaker 3: Do this.

180
00:08:01,840 --> 00:08:03,520
Speaker 2: The challenge says, this is your next step.

181
00:08:03,600 --> 00:08:07,520
Speaker 4: Do this on really simple challenges in programming languages like

182
00:08:07,600 --> 00:08:09,440
Speaker 4: Python that it's got a lot and a lot of

183
00:08:09,480 --> 00:08:13,080
Speaker 4: examples for it does okay past the point where you're

184
00:08:13,160 --> 00:08:16,040
Speaker 4: in the really simple kind of language, things they just

185
00:08:16,240 --> 00:08:18,080
Speaker 4: they sometimes get to the point where they can't even

186
00:08:18,600 --> 00:08:19,840
Speaker 4: create anything that builds it all.

187
00:08:20,680 --> 00:08:24,440
Speaker 1: Huh, why is there? Why does so many engineers swear

188
00:08:24,520 --> 00:08:24,960
Speaker 1: by it? Then?

189
00:08:27,240 --> 00:08:29,640
Speaker 4: Honestly, I'm not sure to what extent the engineers are

190
00:08:29,640 --> 00:08:31,679
Speaker 4: swearing by it. I've talked to a lot of folks

191
00:08:31,680 --> 00:08:35,400
Speaker 4: who are like, you know, my group, you know this

192
00:08:35,640 --> 00:08:39,640
Speaker 4: big bank, you know friend of mine, My group is

193
00:08:39,679 --> 00:08:41,959
Speaker 4: getting copilot gemmed on our throats whether we want it

194
00:08:42,040 --> 00:08:44,559
Speaker 4: or not. And the executives are all really excited about it,

195
00:08:44,760 --> 00:08:47,120
Speaker 4: and none of us are interesting.

196
00:08:47,400 --> 00:08:50,559
Speaker 1: So it's executive. I've I've personally had this theory that

197
00:08:50,600 --> 00:08:54,520
Speaker 1: it's like executive pushed, and that it's all about it's

198
00:08:54,559 --> 00:08:56,800
Speaker 1: all about what the bosses want to see rather than

199
00:08:57,679 --> 00:08:58,079
Speaker 1: even do.

200
00:08:58,840 --> 00:09:02,960
Speaker 2: Sorry, there's a lot of wish fulfillment.

201
00:09:03,040 --> 00:09:04,880
Speaker 4: There's a lot of like, we want to not have

202
00:09:05,000 --> 00:09:07,480
Speaker 4: to deal with these programmers anymore, so we would rather

203
00:09:07,600 --> 00:09:09,360
Speaker 4: deal with the AI thing, and we're just gonna hope

204
00:09:09,400 --> 00:09:11,120
Speaker 4: that the AI thing is going to be, you know,

205
00:09:11,360 --> 00:09:13,000
Speaker 4: just as good as the programmers are close to us,

206
00:09:13,040 --> 00:09:16,199
Speaker 4: just as good as the programmers, and not nearly as annoying.

207
00:09:16,720 --> 00:09:20,559
Speaker 1: Seems like a definitional well maybe that's not the right word.

208
00:09:20,800 --> 00:09:22,959
Speaker 1: Seems like the difference between a software engineer and a

209
00:09:23,000 --> 00:09:26,400
Speaker 1: software developer almost because it's not just about flopping code out,

210
00:09:26,440 --> 00:09:28,240
Speaker 1: it's about making sure the code does stuff.

211
00:09:28,920 --> 00:09:33,719
Speaker 2: Yeah, I mean, those terms get mashed to keep yeah,

212
00:09:33,760 --> 00:09:33,960
Speaker 2: I mean.

213
00:09:34,080 --> 00:09:35,880
Speaker 4: So part of the problem is that in like I

214
00:09:35,960 --> 00:09:37,839
Speaker 4: live in Texas, and in Texas you're not allowed to

215
00:09:37,840 --> 00:09:41,520
Speaker 4: call yourself an engineer unless you passed the engineering exam. Right,

216
00:09:41,679 --> 00:09:45,079
Speaker 4: So I can't literally, I literally can't call myself a

217
00:09:45,120 --> 00:09:47,319
Speaker 4: software engineer legally in Texas As. I understand it. I'm

218
00:09:47,360 --> 00:09:49,520
Speaker 4: not a lawyer, but that's my understanding. So it's like

219
00:09:49,600 --> 00:09:51,439
Speaker 4: the terms get all confused.

220
00:09:51,640 --> 00:09:55,120
Speaker 1: Right, So somewhat related, what is it that people misunderstand

221
00:09:55,160 --> 00:09:55,719
Speaker 1: about the job?

222
00:09:55,840 --> 00:09:58,400
Speaker 4: Then, well, I mean, so one of it is what

223
00:09:58,440 --> 00:10:01,760
Speaker 4: you said earlier, which is that a very small percentage

224
00:10:01,760 --> 00:10:03,959
Speaker 4: of the job is actually slinging code. A lot of

225
00:10:03,960 --> 00:10:05,599
Speaker 4: it is basically trying to figure out what it is

226
00:10:05,640 --> 00:10:08,360
Speaker 4: the code should do based on what you've been told

227
00:10:08,400 --> 00:10:11,120
Speaker 4: that the problem, you know, the solution of the problem

228
00:10:11,120 --> 00:10:11,920
Speaker 4: that you're trying to solve.

229
00:10:13,679 --> 00:10:14,680
Speaker 2: Another thing is that.

230
00:10:16,280 --> 00:10:20,760
Speaker 4: A lot of the problem with the job is that

231
00:10:21,600 --> 00:10:25,559
Speaker 4: every little decision builds up over time, and at some

232
00:10:25,800 --> 00:10:29,400
Speaker 4: point a bug is going to happen. They're inevitable, and

233
00:10:29,520 --> 00:10:33,439
Speaker 4: when that happens, basically there's this process where what you

234
00:10:33,559 --> 00:10:36,640
Speaker 4: need to do, if you're being competent, is roll back

235
00:10:36,679 --> 00:10:40,040
Speaker 4: through the series of decisions, figure out what caused that bug,

236
00:10:40,600 --> 00:10:42,920
Speaker 4: and then figure out what other bugs are likely to

237
00:10:43,000 --> 00:10:45,000
Speaker 4: have been caused by that same set of decisions, and

238
00:10:45,040 --> 00:10:47,920
Speaker 4: then fix not just the bug you're working on, but

239
00:10:48,200 --> 00:10:50,319
Speaker 4: the bugs that you know, not just the bug that's

240
00:10:50,360 --> 00:10:53,280
Speaker 4: been reported, but the bugs that might have also been

241
00:10:53,400 --> 00:10:55,760
Speaker 4: caused by the same problem. Right, And that kind of

242
00:10:55,840 --> 00:10:57,760
Speaker 4: long term thinking is not a thing I've ever seen

243
00:10:58,559 --> 00:11:01,920
Speaker 4: LM exhibit at all. I talk about it like l

244
00:11:02,000 --> 00:11:05,679
Speaker 4: limbs or generative AI is good at solving riddles, but

245
00:11:05,880 --> 00:11:07,880
Speaker 4: actual software development is more like solving a murder.

246
00:11:08,320 --> 00:11:11,480
Speaker 1: Yes, you said that in that wonderful video. Yeah, I

247
00:11:11,880 --> 00:11:16,080
Speaker 1: And it almost feels as if we are building towards

248
00:11:16,120 --> 00:11:19,360
Speaker 1: an actual calamity of sorts, maybe not an immediate one.

249
00:11:19,400 --> 00:11:22,360
Speaker 1: Maybe it'll be kind of sectioned off into areas because

250
00:11:22,360 --> 00:11:25,480
Speaker 1: you've got a new generation of young people coming into

251
00:11:25,520 --> 00:11:28,800
Speaker 1: software engineering or what have you, learning to use AI

252
00:11:29,040 --> 00:11:32,559
Speaker 1: tools rather than your videos definitely talk about this as well,

253
00:11:33,400 --> 00:11:35,960
Speaker 1: actually how to develop software and make sure it works,

254
00:11:36,000 --> 00:11:38,480
Speaker 1: and make sure that it has the infrastructural side inline,

255
00:11:38,559 --> 00:11:41,319
Speaker 1: and also that you're building it with the long term

256
00:11:41,400 --> 00:11:44,240
Speaker 1: thinking of someone else might need to understand how this works,

257
00:11:44,559 --> 00:11:46,760
Speaker 1: and they're not learning that, So you've just got a

258
00:11:46,880 --> 00:11:50,120
Speaker 1: generation of kind of pumping the internet and organizations with

259
00:11:50,600 --> 00:11:51,520
Speaker 1: sloppier code.

260
00:11:52,720 --> 00:11:54,839
Speaker 4: Yes, although I mean one of the problems we're having

261
00:11:54,840 --> 00:11:58,360
Speaker 4: at the moment is that the hiring process for really

262
00:11:58,480 --> 00:12:01,839
Speaker 4: junior engineers is actually pretty broken at the moment, and

263
00:12:01,960 --> 00:12:03,880
Speaker 4: a lot of people are not hiring people that are

264
00:12:03,880 --> 00:12:07,040
Speaker 4: fresh out of school because they're expecting that the AI

265
00:12:07,480 --> 00:12:08,800
Speaker 4: will be able to do.

266
00:12:09,720 --> 00:12:12,600
Speaker 2: Basically, a senior or a mid level developer with.

267
00:12:12,800 --> 00:12:16,280
Speaker 4: The benefit of AI, with the benefit of AI that's

268
00:12:16,280 --> 00:12:19,400
Speaker 4: in air quotes, will be able to do the work

269
00:12:19,600 --> 00:12:22,079
Speaker 4: of that person plus a couple of fresh outs that

270
00:12:22,120 --> 00:12:23,960
Speaker 4: they normally would have hired, but they're not hiring at

271
00:12:23,960 --> 00:12:27,240
Speaker 4: the moment. There's some statistics about how the people that

272
00:12:27,280 --> 00:12:33,240
Speaker 4: are fresh out of school these days are historically underemployed

273
00:12:33,320 --> 00:12:35,720
Speaker 4: relative to the general population, at least in the US

274
00:12:35,760 --> 00:12:36,200
Speaker 4: where I live.

275
00:12:37,240 --> 00:12:39,560
Speaker 1: It also feels like there's no intention behind the code,

276
00:12:40,160 --> 00:12:42,120
Speaker 1: like it's just if you're just generating it. You don't

277
00:12:42,160 --> 00:12:44,320
Speaker 1: really know why you made any patity. You could say

278
00:12:44,400 --> 00:12:47,839
Speaker 1: I chose these lines, But is that at some point

279
00:12:48,040 --> 00:12:51,839
Speaker 1: if you have large amounts of software developers using it,

280
00:12:52,320 --> 00:12:55,240
Speaker 1: however large, But the young people in an organization using

281
00:12:55,280 --> 00:12:58,280
Speaker 1: it to generate their code, they're neither learning to write

282
00:12:58,320 --> 00:13:01,080
Speaker 1: better code, nor are they learning how to just learning

283
00:13:01,120 --> 00:13:04,680
Speaker 1: how to fill in blocks they'll kin within the job.

284
00:13:05,040 --> 00:13:05,640
Speaker 2: Yeah, I mean.

285
00:13:05,800 --> 00:13:08,559
Speaker 4: The the trick is that those of us that have

286
00:13:08,720 --> 00:13:12,120
Speaker 4: spent a whole lot of time debugging software right and

287
00:13:12,240 --> 00:13:14,320
Speaker 4: like finding the problems and digging into them and trying

288
00:13:14,320 --> 00:13:17,920
Speaker 4: to figure out what's going on that kind of stuff.

289
00:13:18,960 --> 00:13:21,599
Speaker 4: It's going to be really hard for younger folks to

290
00:13:21,800 --> 00:13:25,720
Speaker 4: get hired into those jobs so that they have time

291
00:13:25,840 --> 00:13:28,559
Speaker 4: to build the experience to be able to do that.

292
00:13:29,080 --> 00:13:30,880
Speaker 4: And I'm afraid we're going to end up with basically

293
00:13:31,240 --> 00:13:34,520
Speaker 4: an older generation or generations retiring and a newer generation

294
00:13:34,640 --> 00:13:37,360
Speaker 4: that hasn't had the experience of doing that kind of debugging.

295
00:13:38,480 --> 00:13:41,040
Speaker 4: And then it's going to be a real mess, especially

296
00:13:41,120 --> 00:13:44,160
Speaker 4: since from what I can tell, the code that the

297
00:13:44,240 --> 00:13:47,880
Speaker 4: AI's generated are a lot buggier and buggier in weirder

298
00:13:48,240 --> 00:13:51,360
Speaker 4: like randomish kind of ways. Stuff just kind of comes

299
00:13:51,400 --> 00:13:53,000
Speaker 4: out of nowhere in a way that I don't. I mean,

300
00:13:53,280 --> 00:13:55,520
Speaker 4: I've debugged code from people that don't speak the same

301
00:13:55,600 --> 00:13:57,520
Speaker 4: languages as I do, you know, all that kind of

302
00:13:57,559 --> 00:14:01,680
Speaker 4: stuff AI code is different. It's just like, Okay, why

303
00:14:01,800 --> 00:14:04,400
Speaker 4: would anyone want to put that block there that doesn't

304
00:14:04,440 --> 00:14:05,839
Speaker 4: have anything to do with what we're trying to do

305
00:14:05,920 --> 00:14:06,600
Speaker 4: at the moment.

306
00:14:07,040 --> 00:14:10,640
Speaker 1: And why is that? Is it just because it's probabilistic.

307
00:14:10,920 --> 00:14:11,360
Speaker 2: I guess so.

308
00:14:11,960 --> 00:14:14,599
Speaker 4: I mean it's hard to say why. I mean, the

309
00:14:15,320 --> 00:14:17,520
Speaker 4: idea of why an LLN does what it does is

310
00:14:17,640 --> 00:14:20,400
Speaker 4: kind of a you know, anybody's guess.

311
00:14:21,600 --> 00:14:24,960
Speaker 1: Yeah, it's just I keep thinking of the word calamity

312
00:14:25,040 --> 00:14:28,280
Speaker 1: because you sent me these studies as well about how

313
00:14:28,760 --> 00:14:32,040
Speaker 1: they found like a downward pressure on the quality of

314
00:14:32,160 --> 00:14:34,920
Speaker 1: code on GitHub. Would you mind walking me through what

315
00:14:35,040 --> 00:14:35,440
Speaker 1: that means?

316
00:14:35,920 --> 00:14:38,080
Speaker 4: Yeah, So basically what that study found, there were there

317
00:14:38,080 --> 00:14:39,720
Speaker 4: have been a couple of them, but what that particular

318
00:14:39,720 --> 00:14:43,440
Speaker 4: study found is that there's what they call code churn

319
00:14:43,520 --> 00:14:46,480
Speaker 4: has gone up. And code churn is basically when you

320
00:14:46,760 --> 00:14:49,680
Speaker 4: push something you like, add a line of code, you

321
00:14:49,840 --> 00:14:52,840
Speaker 4: push it into two test or to production, and then

322
00:14:53,000 --> 00:14:54,960
Speaker 4: in a short period of time, like I don't remember

323
00:14:55,000 --> 00:14:57,560
Speaker 4: exactly what the definition was, like in a month or

324
00:14:57,600 --> 00:15:01,120
Speaker 4: two months, that line of code changes, right. So basically

325
00:15:01,200 --> 00:15:03,000
Speaker 4: what that means is that the line of code that

326
00:15:03,120 --> 00:15:08,240
Speaker 4: got created, somebody decided after it got put in, oh wait, no,

327
00:15:08,400 --> 00:15:10,160
Speaker 4: that doesn't work right, we don't We're not happy with that.

328
00:15:10,200 --> 00:15:12,080
Speaker 4: We're going to change it to be something else, right, right,

329
00:15:12,600 --> 00:15:15,080
Speaker 4: And the percentage of lines or the number of lines

330
00:15:15,280 --> 00:15:20,440
Speaker 4: that that get changed fairly quickly after they get submitted,

331
00:15:21,040 --> 00:15:26,120
Speaker 4: has gone way up since the since the implementation of

332
00:15:26,440 --> 00:15:30,120
Speaker 4: get hub copilot. So and this is across like most

333
00:15:30,200 --> 00:15:32,800
Speaker 4: of the giant you know, millions of lines of codes

334
00:15:32,840 --> 00:15:33,760
Speaker 4: on GitHub and.

335
00:15:33,840 --> 00:15:37,280
Speaker 1: For a simpleton me, why does it being changed? Why

336
00:15:37,400 --> 00:15:38,960
Speaker 1: is changing it so often bad?

337
00:15:39,960 --> 00:15:43,000
Speaker 2: Like? Well, I mean, so, I mean if you do

338
00:15:43,080 --> 00:15:44,640
Speaker 2: it right the first time, you can move on to

339
00:15:44,720 --> 00:15:45,240
Speaker 2: the next thing.

340
00:15:45,640 --> 00:15:45,840
Speaker 3: Ah.

341
00:15:46,360 --> 00:15:46,520
Speaker 1: Right.

342
00:15:46,920 --> 00:15:49,120
Speaker 4: If it's like you know, if you're writing a document

343
00:15:49,240 --> 00:15:51,320
Speaker 4: and you put put the document in there and then

344
00:15:51,360 --> 00:15:54,680
Speaker 4: you like you're get you're in in Google Docs and

345
00:15:54,720 --> 00:15:57,200
Speaker 4: you're like tracking changes and it's like, Okay, this sentence

346
00:15:57,240 --> 00:15:58,440
Speaker 4: has changed seventeen times.

347
00:15:58,480 --> 00:16:01,280
Speaker 2: Obviously the person isn't happy with that, right.

348
00:16:01,520 --> 00:16:04,240
Speaker 1: So the generative code isn't good, right, and so people

349
00:16:04,280 --> 00:16:05,200
Speaker 1: see you need to change.

350
00:16:05,560 --> 00:16:07,560
Speaker 2: That's the presumption, yes, and.

351
00:16:08,960 --> 00:16:11,600
Speaker 1: So I it also said the code quality itself? Is

352
00:16:11,640 --> 00:16:14,200
Speaker 1: that the only way they is that the only way

353
00:16:14,240 --> 00:16:16,400
Speaker 1: they measured it? Or is it there were other things

354
00:16:16,440 --> 00:16:18,160
Speaker 1: as well, So.

355
00:16:18,320 --> 00:16:21,520
Speaker 2: They measured that they measured.

356
00:16:23,120 --> 00:16:25,720
Speaker 1: Uh like moved code.

357
00:16:26,040 --> 00:16:27,520
Speaker 2: Yeah, that the move code.

358
00:16:27,800 --> 00:16:31,360
Speaker 4: The thing I was talking about earlier, where the uh

359
00:16:31,600 --> 00:16:33,960
Speaker 4: you've got a bunch of different places in the code

360
00:16:34,240 --> 00:16:38,000
Speaker 4: that all do the same You try to do the

361
00:16:38,040 --> 00:16:39,000
Speaker 4: same function, but they do it.

362
00:16:39,000 --> 00:16:42,520
Speaker 2: In different ways. Normally, what would happen is you'd have

363
00:16:42,760 --> 00:16:43,400
Speaker 2: your you do it.

364
00:16:43,480 --> 00:16:45,880
Speaker 4: You do a thing here, right, and then at some

365
00:16:45,960 --> 00:16:47,360
Speaker 4: point in the future you need to do that thing

366
00:16:47,480 --> 00:16:49,320
Speaker 4: again in a different place. And so what you do

367
00:16:49,480 --> 00:16:52,280
Speaker 4: is you would move that original block that does the

368
00:16:52,360 --> 00:16:55,280
Speaker 4: thing someplace else, and then you would call that block

369
00:16:55,320 --> 00:16:57,800
Speaker 4: from both places because it already works, right, And then

370
00:16:57,880 --> 00:17:00,040
Speaker 4: that way you've got you know, however, you go that

371
00:17:00,160 --> 00:17:01,960
Speaker 4: stuff from the server, you're fetching it the same way,

372
00:17:03,360 --> 00:17:06,080
Speaker 4: but with this thing. Basically instead of doing that, you've

373
00:17:06,080 --> 00:17:08,199
Speaker 4: got copy paste. Okay, when we put another one here,

374
00:17:08,240 --> 00:17:09,680
Speaker 4: and we put another one here, let me put another

375
00:17:09,720 --> 00:17:11,560
Speaker 4: one here, and it's a it's a maintenance nightmare.

376
00:17:23,560 --> 00:17:26,719
Speaker 1: So for the for the audience as well, how does

377
00:17:26,760 --> 00:17:29,720
Speaker 1: the software developer actually use GitHub? Like really simple stuff?

378
00:17:29,720 --> 00:17:31,560
Speaker 1: I realized, but I think it's important for people to

379
00:17:31,840 --> 00:17:33,480
Speaker 1: It just occurred to be like, this may be something

380
00:17:33,520 --> 00:17:35,560
Speaker 1: that most listeners don't know, which is good to I

381
00:17:35,600 --> 00:17:36,120
Speaker 1: think it's good.

382
00:17:36,320 --> 00:17:36,560
Speaker 2: Yeah.

383
00:17:36,960 --> 00:17:40,000
Speaker 4: So so what we do is we basically make changes

384
00:17:40,040 --> 00:17:43,680
Speaker 4: to code. We get to the point where we the developer,

385
00:17:43,720 --> 00:17:46,439
Speaker 4: are happy with the way it's set up on our machine,

386
00:17:46,520 --> 00:17:47,920
Speaker 4: and then we do what it's called a push, and

387
00:17:48,040 --> 00:17:50,200
Speaker 4: we basically send all that code, submit all that code

388
00:17:50,280 --> 00:17:54,520
Speaker 4: up to get hub, and then theoretically, you know, there

389
00:17:54,560 --> 00:17:57,200
Speaker 4: can be automatic processes that kick in that like check

390
00:17:57,280 --> 00:17:59,359
Speaker 4: that code for particular things and run tests on and

391
00:17:59,400 --> 00:18:01,639
Speaker 4: that kind of stuff. And then at some point we

392
00:18:01,720 --> 00:18:03,840
Speaker 4: have a thing called a poor request, which is basically

393
00:18:03,880 --> 00:18:05,280
Speaker 4: a thing that says, Okay, I would like this to

394
00:18:05,359 --> 00:18:08,160
Speaker 4: go into production now, or more or less, I would

395
00:18:08,240 --> 00:18:10,919
Speaker 4: like this to get promoted into the next phase now,

396
00:18:11,080 --> 00:18:13,320
Speaker 4: and then someone theoretically will look at it and go okay,

397
00:18:13,400 --> 00:18:15,600
Speaker 4: that's fine, and then click the yes button or say, hey,

398
00:18:15,960 --> 00:18:17,639
Speaker 4: you forgot about this, go look at this or that

399
00:18:17,720 --> 00:18:20,320
Speaker 4: kind of thing, right, And the poor requests is kind

400
00:18:20,359 --> 00:18:22,159
Speaker 4: of the unit of work kind of.

401
00:18:23,920 --> 00:18:25,600
Speaker 1: So with get hub you almost use it like an

402
00:18:25,720 --> 00:18:30,800
Speaker 1: organizational code dump, centralize all the code. Sorry just for

403
00:18:30,880 --> 00:18:35,320
Speaker 1: the non coding as well. And I think it's I

404
00:18:35,440 --> 00:18:38,360
Speaker 1: think that the LLLM industry has done a really good

405
00:18:38,480 --> 00:18:41,280
Speaker 1: job of dancing around these terms and selling them to

406
00:18:41,320 --> 00:18:44,080
Speaker 1: people like me. While they weren't selling they didn't work

407
00:18:44,119 --> 00:18:47,120
Speaker 1: on me. I am too stupid, But it's where they've

408
00:18:47,240 --> 00:18:49,680
Speaker 1: just like been like, okay, yeah, well lots of people

409
00:18:49,840 --> 00:18:52,680
Speaker 1: use co pilot, that's good, and this is good because

410
00:18:53,040 --> 00:18:56,760
Speaker 1: software is coding. But it kind of feels like, I

411
00:18:56,840 --> 00:18:59,400
Speaker 1: don't know, all of this is taking the one thing,

412
00:18:59,800 --> 00:19:04,159
Speaker 1: like one major pot out of software development and ruining it.

413
00:19:04,320 --> 00:19:06,240
Speaker 1: And I don't even mean coding. I mean it's the

414
00:19:06,359 --> 00:19:11,920
Speaker 1: intentionality behind software design and infrastructure and maintain Like there's

415
00:19:12,200 --> 00:19:16,159
Speaker 1: it seems like they're removing intention in multiple parts.

416
00:19:19,680 --> 00:19:21,600
Speaker 4: So the way I would say it is when they

417
00:19:21,720 --> 00:19:23,760
Speaker 4: talk about the AI being able to do the work

418
00:19:23,760 --> 00:19:27,239
Speaker 4: of a programmer, what they're doing is they're devaluing all

419
00:19:27,280 --> 00:19:32,119
Speaker 4: of the stuff that's not just hacking code, right, And

420
00:19:32,240 --> 00:19:34,080
Speaker 4: so what they're saying is that basically the job of

421
00:19:34,119 --> 00:19:38,800
Speaker 4: a developer is basically just you know, typing, basically, right,

422
00:19:39,200 --> 00:19:40,840
Speaker 4: And that all of the work that we do to

423
00:19:41,040 --> 00:19:43,600
Speaker 4: understand what the problem actually is and how it needs

424
00:19:43,640 --> 00:19:47,120
Speaker 4: to work, and you know what other problems are likely

425
00:19:47,200 --> 00:19:49,040
Speaker 4: to show up when we try to do that, and

426
00:19:49,119 --> 00:19:51,200
Speaker 4: how to avoid those things as we go and that

427
00:19:51,320 --> 00:19:54,200
Speaker 4: kind of thing, all that work is basically not important.

428
00:19:56,200 --> 00:20:01,560
Speaker 1: And I mean I two words which would probably annoy

429
00:20:01,560 --> 00:20:04,399
Speaker 1: you this is I feel like vibe coating is the

430
00:20:04,520 --> 00:20:08,000
Speaker 1: other part of this. So if I'm correct, correct me

431
00:20:08,000 --> 00:20:10,680
Speaker 1: if I'm wrong, vibe coating is just typing stuff into

432
00:20:10,680 --> 00:20:13,080
Speaker 1: an LEM and software comes out and hopefully it works.

433
00:20:13,520 --> 00:20:18,359
Speaker 2: Yeah. Vibe coating is basically when you intentionally try it.

434
00:20:18,480 --> 00:20:21,360
Speaker 4: Well, I don't know, but intentionally, but basically you make

435
00:20:21,400 --> 00:20:23,840
Speaker 4: a point of not digging into the code and looking

436
00:20:23,840 --> 00:20:26,440
Speaker 4: at what the LM is doing, and you basically say, Okay,

437
00:20:26,640 --> 00:20:28,920
Speaker 4: I would like something that does X right. I would

438
00:20:29,000 --> 00:20:32,200
Speaker 4: like a game where I fly airplanes around a city

439
00:20:32,359 --> 00:20:35,720
Speaker 4: or something right, And then you get what it spits out,

440
00:20:35,920 --> 00:20:38,600
Speaker 4: and then you say, you know, okay, let me try it. Okay, well,

441
00:20:38,800 --> 00:20:40,760
Speaker 4: can we have more airplanes? And okay, can we have

442
00:20:40,840 --> 00:20:42,800
Speaker 4: some balloons with you know, signs on them now? And

443
00:20:42,880 --> 00:20:44,520
Speaker 4: can we do this kind of thing? And then you

444
00:20:44,680 --> 00:20:47,200
Speaker 4: don't think about what the side effects are. You don't

445
00:20:47,240 --> 00:20:49,639
Speaker 4: think about what things could go wrong, You don't think

446
00:20:49,680 --> 00:20:51,879
Speaker 4: about air conditions that kind of stuff, and you just

447
00:20:52,000 --> 00:20:55,600
Speaker 4: hope that this whatever you look at and has the

448
00:20:55,720 --> 00:20:58,320
Speaker 4: right vibe and that you know, if it if it

449
00:20:58,400 --> 00:21:01,040
Speaker 4: looks like kind of what you wanted, that probably it's

450
00:21:01,080 --> 00:21:02,880
Speaker 4: going to be fine, or hopefully it's going to be fine.

451
00:21:03,240 --> 00:21:04,680
Speaker 1: How do you feel about vibe coating?

452
00:21:05,800 --> 00:21:06,720
Speaker 3: So I do it.

453
00:21:06,800 --> 00:21:09,320
Speaker 4: Sometimes vicoding is great for a thing that you're going

454
00:21:09,400 --> 00:21:11,040
Speaker 4: to do once and then throw away.

455
00:21:12,040 --> 00:21:12,280
Speaker 1: Yeah.

456
00:21:12,520 --> 00:21:14,280
Speaker 4: Right, So if it's like, you know, okay, I want

457
00:21:14,280 --> 00:21:15,560
Speaker 4: to I want to do a thing. I want to

458
00:21:15,600 --> 00:21:17,840
Speaker 4: translate this thing to you know, I want to make

459
00:21:17,920 --> 00:21:19,720
Speaker 4: this table go into this format over here or that

460
00:21:19,800 --> 00:21:21,439
Speaker 4: kind of thing. You do it, you get the output

461
00:21:21,480 --> 00:21:23,320
Speaker 4: you want, you throw the code away. No big deal, right,

462
00:21:23,400 --> 00:21:27,639
Speaker 4: like a prototype almost, yeah, basically, And so you know,

463
00:21:28,080 --> 00:21:30,720
Speaker 4: we call them spikes or tracer bullets. Sometimes it's like

464
00:21:30,760 --> 00:21:33,159
Speaker 4: a you know, let me get a thing that works

465
00:21:33,880 --> 00:21:36,080
Speaker 4: at all, right, and then let me see what I

466
00:21:36,119 --> 00:21:39,640
Speaker 4: can learn from that to move into my big maintainable project.

467
00:21:40,840 --> 00:21:43,600
Speaker 4: But for anything that's like, you know, this thing needs

468
00:21:43,640 --> 00:21:45,359
Speaker 4: to run for a while, this thing's needs to not

469
00:21:45,480 --> 00:21:46,000
Speaker 4: get hacked.

470
00:21:46,119 --> 00:21:49,000
Speaker 2: This thing needs to you know, not crash. It's a

471
00:21:49,040 --> 00:21:49,760
Speaker 2: really bad idea.

472
00:21:50,520 --> 00:21:53,760
Speaker 1: Yeah, And at some point I feel like someone building

473
00:21:53,800 --> 00:21:56,240
Speaker 1: a product that they don't really understand the working So

474
00:21:56,240 --> 00:21:59,760
Speaker 1: of it's kind of almost identical to generating a story

475
00:21:59,840 --> 00:22:03,600
Speaker 1: with at GPT, except kind of more complex and more

476
00:22:03,680 --> 00:22:04,480
Speaker 1: prone to errors.

477
00:22:05,320 --> 00:22:05,520
Speaker 3: Yeah.

478
00:22:06,760 --> 00:22:10,040
Speaker 4: And the other thing is that there's an adversarial component, right,

479
00:22:10,800 --> 00:22:14,959
Speaker 4: so people will intentionally try to go hack that thing

480
00:22:15,000 --> 00:22:17,560
Speaker 4: that's sitting on the internet, right, Yeah, in a way

481
00:22:17,600 --> 00:22:19,560
Speaker 4: that they don't intentionally try to go mess with the

482
00:22:19,600 --> 00:22:23,000
Speaker 4: story that you wrote, right, right, And so even if

483
00:22:23,080 --> 00:22:25,240
Speaker 4: it works all by itself, that doesn't mean it's going

484
00:22:25,320 --> 00:22:27,480
Speaker 4: to work when somebody starts pounding on it intentionally trying

485
00:22:27,520 --> 00:22:30,240
Speaker 4: to break it. And if they can break it, then

486
00:22:30,320 --> 00:22:32,359
Speaker 4: that's a whole other set of problems that you now have.

487
00:22:33,160 --> 00:22:36,159
Speaker 1: It feels like quality assurance is just never part. Oh no,

488
00:22:36,280 --> 00:22:38,920
Speaker 1: they are they claiming they're going to do quality assurance

489
00:22:39,000 --> 00:22:41,399
Speaker 1: with large language models. Yet they must some.

490
00:22:41,480 --> 00:22:44,680
Speaker 4: People are Yeah, I mean, to be honest, a lot

491
00:22:44,760 --> 00:22:48,960
Speaker 4: of companies have just been getting rid of quality assurants

492
00:22:49,119 --> 00:22:49,720
Speaker 4: over the years.

493
00:22:49,840 --> 00:22:49,959
Speaker 1: Right.

494
00:22:50,240 --> 00:22:52,480
Speaker 4: Really, when I worked at IBM, we didn't have quality

495
00:22:52,480 --> 00:22:56,200
Speaker 4: assurance at all. They would, no, seriously, they would do this.

496
00:22:56,359 --> 00:22:58,000
Speaker 4: I was in IBM's Cloud group and they would do

497
00:22:58,080 --> 00:23:02,000
Speaker 4: these these what do they call them, uh, packathon kind

498
00:23:02,040 --> 00:23:03,000
Speaker 4: of things that they didn't call that.

499
00:23:03,040 --> 00:23:03,720
Speaker 2: I don't know what they called it.

500
00:23:03,800 --> 00:23:07,200
Speaker 4: But basically everybody in all the other development groups would

501
00:23:07,200 --> 00:23:09,680
Speaker 4: get together and basically bang on the code that was

502
00:23:09,720 --> 00:23:11,879
Speaker 4: about to get released from some other group to try

503
00:23:11,920 --> 00:23:13,840
Speaker 4: to see if they could break it. Right, But they

504
00:23:13,880 --> 00:23:16,680
Speaker 4: didn't have dedicated testers anymore because they decided I guess

505
00:23:16,760 --> 00:23:19,280
Speaker 4: that they didn't think they were worth the money.

506
00:23:19,359 --> 00:23:22,280
Speaker 2: I don't know, but we had some issues because of that.

507
00:23:22,800 --> 00:23:24,520
Speaker 1: When spinal movement happened.

508
00:23:25,760 --> 00:23:27,560
Speaker 4: I was in I don't know, so I was at

509
00:23:27,760 --> 00:23:31,639
Speaker 4: IBM in like twenty seventeen, twenty eighteen, right, so it

510
00:23:31,680 --> 00:23:32,800
Speaker 4: would have been sometime prior to that.

511
00:23:32,880 --> 00:23:34,840
Speaker 2: When I got there, they didn't have any QA folks.

512
00:23:35,480 --> 00:23:38,520
Speaker 1: Really just feels like the it's the management problem as well.

513
00:23:38,640 --> 00:23:40,679
Speaker 1: It's the management cotton people.

514
00:23:40,920 --> 00:23:41,399
Speaker 2: I would think.

515
00:23:41,480 --> 00:23:46,240
Speaker 1: So it's a real shame as well. And I forgive

516
00:23:46,320 --> 00:23:49,199
Speaker 1: me if I'm forgetting exactly where You've mentioned as well

517
00:23:49,240 --> 00:23:52,480
Speaker 1: that there is like compound scar tissue from AI generated code,

518
00:23:52,560 --> 00:23:55,879
Speaker 1: a larger problem of lots of this code being generated

519
00:23:55,920 --> 00:23:56,280
Speaker 1: with AI.

520
00:23:57,160 --> 00:24:03,840
Speaker 2: Well that's that's my expert, right, Yeah, just a potential worry, right.

521
00:24:03,840 --> 00:24:05,840
Speaker 4: Right, so that the more of this we get and

522
00:24:05,960 --> 00:24:09,520
Speaker 4: the more issues that we have, the more stuff we're

523
00:24:09,520 --> 00:24:11,600
Speaker 4: gonna have to dig out of, right, And what I'm

524
00:24:11,680 --> 00:24:14,000
Speaker 4: honestly envisioning at some point in the I don't know

525
00:24:14,080 --> 00:24:14,800
Speaker 4: how long it will take.

526
00:24:14,880 --> 00:24:17,240
Speaker 2: The crypto bubble took way longer to pop than I expected.

527
00:24:17,280 --> 00:24:18,399
Speaker 4: So I don't know how long it's going to be

528
00:24:18,480 --> 00:24:21,479
Speaker 4: before this one does, but I'm expecting that there's going

529
00:24:21,560 --> 00:24:23,679
Speaker 4: to be this big push to try to clean up

530
00:24:23,680 --> 00:24:26,120
Speaker 4: a bunch of this crap here in a few years,

531
00:24:26,160 --> 00:24:28,280
Speaker 4: once people realize that a lot of the code that's

532
00:24:28,280 --> 00:24:31,960
Speaker 4: being written and generated right now is has all of

533
00:24:32,000 --> 00:24:35,520
Speaker 4: these vulnerabilities that nobody's bothering to check for at the moment.

534
00:24:35,920 --> 00:24:39,800
Speaker 1: Right, and those vulnerabilities again non technical way, I read

535
00:24:39,840 --> 00:24:42,320
Speaker 1: that it was like the call upon things on GitHub

536
00:24:42,400 --> 00:24:45,400
Speaker 1: that don't exist, so bad actors create something that resembles

537
00:24:45,480 --> 00:24:46,240
Speaker 1: what it's pulling from.

538
00:24:46,600 --> 00:24:50,080
Speaker 4: That's so that's that's a more specific kind of one.

539
00:24:50,240 --> 00:24:52,159
Speaker 4: I mean, there are a lot of things. I mean,

540
00:24:52,760 --> 00:24:55,600
Speaker 4: so there have been computer viruses since the eighties, right right,

541
00:24:56,359 --> 00:24:58,160
Speaker 4: you know, the Morris worm and that kind of stuff.

542
00:24:58,160 --> 00:25:03,560
Speaker 4: And basically there are no own ways that code if

543
00:25:03,600 --> 00:25:05,639
Speaker 4: you have to write it in a particular way in

544
00:25:05,760 --> 00:25:07,800
Speaker 4: order for it to be secure, right right, And even

545
00:25:07,880 --> 00:25:10,320
Speaker 4: then sometimes people come up with novel ways of making

546
00:25:10,400 --> 00:25:11,639
Speaker 4: something not secure, and.

547
00:25:11,720 --> 00:25:13,159
Speaker 1: How how do you have to write it to make

548
00:25:13,200 --> 00:25:15,600
Speaker 1: it secure? If it's possible to explain.

549
00:25:15,400 --> 00:25:17,600
Speaker 4: Well, I mean, there's a big, long list of rules, right.

550
00:25:17,720 --> 00:25:19,320
Speaker 4: I mean, one thing you can do is you can

551
00:25:19,440 --> 00:25:22,960
Speaker 4: use languages that are what they call safer. But still

552
00:25:23,040 --> 00:25:24,879
Speaker 4: you have to make sure that any input that you

553
00:25:25,040 --> 00:25:27,479
Speaker 4: get from the network, you're really really careful to make

554
00:25:27,480 --> 00:25:30,560
Speaker 4: sure that it doesn't get to overwrite parts of your

555
00:25:30,640 --> 00:25:34,360
Speaker 4: program that actually execute things. You have to make sure

556
00:25:34,400 --> 00:25:37,040
Speaker 4: that it doesn't have the opportunity to be able to

557
00:25:37,080 --> 00:25:39,119
Speaker 4: write to places on your disc that it shouldn't be

558
00:25:39,160 --> 00:25:42,080
Speaker 4: able to write to. You have to be able to

559
00:25:42,080 --> 00:25:44,280
Speaker 4: make sure that it doesn't have access to read data

560
00:25:44,320 --> 00:25:46,280
Speaker 4: that it shouldn't be able to read, you know, all

561
00:25:46,359 --> 00:25:48,560
Speaker 4: that kind of stuff. And when those things don't happen,

562
00:25:48,720 --> 00:25:52,480
Speaker 4: you end up with you know, so and so got hacked.

563
00:25:52,960 --> 00:25:56,080
Speaker 4: You know, turns out that somebody, we think maybe China,

564
00:25:56,200 --> 00:25:59,800
Speaker 4: is reading the email of the you know, people in Congress.

565
00:26:01,600 --> 00:26:03,560
Speaker 4: You get another letter in the mail that says your

566
00:26:03,600 --> 00:26:07,800
Speaker 4: social Security number has been you know, leaked by you know,

567
00:26:07,960 --> 00:26:10,440
Speaker 4: some credit checking firm or something like that.

568
00:26:10,840 --> 00:26:12,840
Speaker 1: Even even like I think it was what the big

569
00:26:12,920 --> 00:26:15,920
Speaker 1: hot target data breach from a while back was through

570
00:26:15,920 --> 00:26:20,479
Speaker 1: the HVAC system. It was it was it's just except

571
00:26:20,520 --> 00:26:23,399
Speaker 1: now we've got and that was with humans writing the

572
00:26:23,720 --> 00:26:27,800
Speaker 1: code right, imagine if we didn't know. Oh god, it

573
00:26:27,880 --> 00:26:31,600
Speaker 1: really does feel like the young people are going like that. Actually, no,

574
00:26:31,680 --> 00:26:34,240
Speaker 1: I take it back. You were talking about agile the

575
00:26:34,280 --> 00:26:35,760
Speaker 1: other day. I'm going to ask you to explain that

576
00:26:35,800 --> 00:26:38,760
Speaker 1: in a second. But it's like, it sounds like for

577
00:26:39,160 --> 00:26:42,680
Speaker 1: almost decades they've been gnawing away at management's been gnawing

578
00:26:42,720 --> 00:26:45,520
Speaker 1: away at the sides of building good software and building

579
00:26:45,560 --> 00:26:46,600
Speaker 1: good software culture.

580
00:26:48,920 --> 00:26:51,399
Speaker 4: Yes, I mean, there's an argument that says we never

581
00:26:51,440 --> 00:26:52,760
Speaker 4: got it right in the first place. But I mean,

582
00:26:53,520 --> 00:26:55,600
Speaker 4: I mean, if you think about it, software has been

583
00:26:55,640 --> 00:26:59,600
Speaker 4: a thing for what fifty years, sixty years, seventy years, right,

584
00:26:59,680 --> 00:27:02,760
Speaker 4: I mean compare that to like construction engineering or bridge

585
00:27:02,800 --> 00:27:05,159
Speaker 4: building or that kind of stuff. Right, we're still, you know,

586
00:27:05,480 --> 00:27:08,159
Speaker 4: relatively speaking, in the infancy, In our infancy as a

587
00:27:08,280 --> 00:27:12,119
Speaker 4: as an industry. You know, it's been a it's been

588
00:27:12,160 --> 00:27:15,040
Speaker 4: a constant evolution, and a lot of times the things

589
00:27:15,160 --> 00:27:19,560
Speaker 4: that were the things that we did to solve a

590
00:27:19,680 --> 00:27:21,840
Speaker 4: problem that we had ended up causing other problems.

591
00:27:21,960 --> 00:27:22,080
Speaker 1: Right.

592
00:27:22,600 --> 00:27:25,639
Speaker 4: So, going back to agile, in the long long ago, right,

593
00:27:25,840 --> 00:27:28,040
Speaker 4: we used to manage software projects the same way we

594
00:27:28,160 --> 00:27:31,800
Speaker 4: manage like build you know, bridge building and building building project,

595
00:27:32,240 --> 00:27:35,720
Speaker 4: you know, construction projects. And it turns out that when

596
00:27:35,800 --> 00:27:39,320
Speaker 4: you're going to build a bridge, you know beforehand what

597
00:27:39,520 --> 00:27:41,400
Speaker 4: you need to build a bridge to do. When you're

598
00:27:41,400 --> 00:27:43,679
Speaker 4: building software, a lot of times people are changing their

599
00:27:43,680 --> 00:27:45,639
Speaker 4: minds as you go. Right, and you build a thing

600
00:27:45,680 --> 00:27:46,959
Speaker 4: and you show it to them and they're like, oh,

601
00:27:47,200 --> 00:27:48,560
Speaker 4: why don't we put this over here, and why don't

602
00:27:48,560 --> 00:27:50,280
Speaker 4: we change this? And that kind of thing right right,

603
00:27:50,560 --> 00:27:53,040
Speaker 4: because you don't have the same kind of constraints physical

604
00:27:53,040 --> 00:27:54,720
Speaker 4: constraints that you do when you're trying to build a bridge.

605
00:27:55,040 --> 00:27:58,359
Speaker 4: And so we gotten this problem where you would create

606
00:27:58,400 --> 00:28:00,119
Speaker 4: these project plans about how you were going to to

607
00:28:00,160 --> 00:28:01,960
Speaker 4: build this thing, and you would never be anywhere close

608
00:28:02,000 --> 00:28:04,000
Speaker 4: to on time because things would change the whole time.

609
00:28:04,800 --> 00:28:07,320
Speaker 4: And so they created this thing called the agile methodology.

610
00:28:07,480 --> 00:28:10,439
Speaker 4: I'm drastically simplifying. There were steps in the middle, but basically,

611
00:28:10,560 --> 00:28:15,240
Speaker 4: so this agile thing is where we instead of saying, okay,

612
00:28:15,280 --> 00:28:16,720
Speaker 4: so this is what the whole project's going to look like,

613
00:28:16,720 --> 00:28:18,879
Speaker 4: we're gonna be doing. We're going to be done in

614
00:28:18,920 --> 00:28:20,879
Speaker 4: six months, and then things changing along the way, we

615
00:28:21,000 --> 00:28:23,080
Speaker 4: basically block off a thing called a sprint. It's a

616
00:28:23,119 --> 00:28:25,480
Speaker 4: week or two or a month, maybe it depends. And

617
00:28:25,600 --> 00:28:27,840
Speaker 4: then you know, everybody picks their own sprint length and

618
00:28:27,880 --> 00:28:29,960
Speaker 4: then you go, okay, I'm only going to talk about

619
00:28:30,040 --> 00:28:32,439
Speaker 4: what's going to happen in the next sprint or two, right,

620
00:28:32,560 --> 00:28:33,800
Speaker 4: And then you get to the end of that two

621
00:28:33,800 --> 00:28:35,359
Speaker 4: weeks and you go, okay, cool, this is what we

622
00:28:35,440 --> 00:28:37,119
Speaker 4: got done. What do we want to do next? And

623
00:28:37,160 --> 00:28:38,560
Speaker 4: then okay, that's what we got done, and what do

624
00:28:38,640 --> 00:28:39,800
Speaker 4: we want to do next? And that kind of thing,

625
00:28:40,000 --> 00:28:43,200
Speaker 4: and that way, as you go, you have the opportunity

626
00:28:43,240 --> 00:28:46,080
Speaker 4: to change things. You have an opportunity to roll changes

627
00:28:46,200 --> 00:28:48,880
Speaker 4: into the process, that kind of thing. Right. The problem

628
00:28:48,960 --> 00:28:52,080
Speaker 4: with that is kind of the same way that dates

629
00:28:52,080 --> 00:28:55,520
Speaker 4: always ran out in water in waterfall, land projects can

630
00:28:55,600 --> 00:28:57,880
Speaker 4: go way way longer than they were expected to at

631
00:28:57,880 --> 00:29:00,479
Speaker 4: the beginning because everybody's focused on just two weeks at

632
00:29:00,480 --> 00:29:02,280
Speaker 4: a time, and you never kind of take a big

633
00:29:02,360 --> 00:29:04,880
Speaker 4: step back like you ought to and go, okay, wait,

634
00:29:05,160 --> 00:29:07,480
Speaker 4: you know we were supposed to be done, you know,

635
00:29:07,720 --> 00:29:08,400
Speaker 4: two months ago.

636
00:29:09,320 --> 00:29:10,520
Speaker 2: When are we going to wrap this up?

637
00:29:11,640 --> 00:29:14,840
Speaker 1: Right? And how has that led to things agting worse?

638
00:29:14,960 --> 00:29:17,560
Speaker 1: Is it that just software culture software development culture has

639
00:29:17,600 --> 00:29:21,560
Speaker 1: been focused on short term perpetually.

640
00:29:21,200 --> 00:29:23,160
Speaker 2: Is the short term is part of it.

641
00:29:23,800 --> 00:29:27,800
Speaker 4: Part of it is there are you know, unscrupulous developers

642
00:29:27,840 --> 00:29:31,480
Speaker 4: out there that basically want to extend the length of

643
00:29:31,560 --> 00:29:33,160
Speaker 4: the project so they can get more money out of.

644
00:29:33,160 --> 00:29:36,920
Speaker 2: It, right right, That's that's always the case. But the

645
00:29:37,000 --> 00:29:39,800
Speaker 2: other thing is that you end up with a real.

646
00:29:42,200 --> 00:29:43,400
Speaker 4: A lot of times, you end up with a real

647
00:29:43,560 --> 00:29:46,360
Speaker 4: lack of like long term planning and long term understanding,

648
00:29:46,840 --> 00:29:48,080
Speaker 4: right right, because everybody's you.

649
00:29:48,080 --> 00:29:50,920
Speaker 2: Know, some kind of thing. You know, companies are only

650
00:29:50,960 --> 00:29:52,280
Speaker 2: worried about what happens next quarter.

651
00:29:52,440 --> 00:29:52,520
Speaker 1: Right.

652
00:29:52,960 --> 00:29:54,400
Speaker 4: If you're only worried about what's going to happen the

653
00:29:54,440 --> 00:29:57,920
Speaker 4: next week or the next four weeks, the things that

654
00:29:58,000 --> 00:30:01,480
Speaker 4: you look on, look at, you know, tends not to

655
00:30:01,680 --> 00:30:04,600
Speaker 4: have the longer term implications that sometimes you need. Right

656
00:30:05,000 --> 00:30:06,480
Speaker 4: And there are times you get close to the end

657
00:30:06,600 --> 00:30:10,000
Speaker 4: and you're like, oh, you know, we didn't think about

658
00:30:10,040 --> 00:30:11,800
Speaker 4: this problem yeaheah.

659
00:30:12,560 --> 00:30:15,240
Speaker 1: And also if you're in a two or three week thing,

660
00:30:15,480 --> 00:30:18,880
Speaker 1: you're probably not thinking even what you did last sprint like.

661
00:30:18,960 --> 00:30:22,520
Speaker 2: It maybe last one, but not like two or three,

662
00:30:22,840 --> 00:30:23,760
Speaker 2: two or three ones ago.

663
00:30:24,320 --> 00:30:27,320
Speaker 1: Is this a problem throughout organizations of all sizes? Is

664
00:30:27,360 --> 00:30:30,200
Speaker 1: this a consultancy problem. Is it everywhere?

665
00:30:31,800 --> 00:30:35,800
Speaker 2: It's most places, Huh, there are, there are some places

666
00:30:35,880 --> 00:30:36,280
Speaker 2: that are.

667
00:30:38,560 --> 00:30:44,280
Speaker 4: Usually in startups, we're a lot more ad hoc and

668
00:30:44,400 --> 00:30:47,760
Speaker 4: we're a lot more you know, focused on trying to

669
00:30:47,800 --> 00:30:52,440
Speaker 4: get things done. The basically, the the the idea is

670
00:30:52,680 --> 00:30:56,280
Speaker 4: the larger you get as an organization, and the more

671
00:30:56,360 --> 00:30:59,000
Speaker 4: money you're throwing at it, and the more management control

672
00:30:59,120 --> 00:31:02,640
Speaker 4: you want, the more of this overhead you put in place,

673
00:31:02,720 --> 00:31:05,000
Speaker 4: and the more complicated things get as just as a

674
00:31:05,080 --> 00:31:06,479
Speaker 4: as a management structure kind of thing.

675
00:31:07,400 --> 00:31:09,760
Speaker 1: And this in the big So this is something you'd

676
00:31:09,800 --> 00:31:11,640
Speaker 1: seem like a Google and an Amazon as well.

677
00:31:11,760 --> 00:31:13,560
Speaker 2: Oh absolutely so.

678
00:31:13,760 --> 00:31:16,880
Speaker 1: Do you do you think it has the same organizational effects.

679
00:31:16,600 --> 00:31:20,720
Speaker 2: Or largely yes.

680
00:31:21,640 --> 00:31:27,920
Speaker 4: The so those organizations tend to be well, those organizations

681
00:31:28,080 --> 00:31:29,560
Speaker 4: historically have tended to be.

682
00:31:31,080 --> 00:31:33,840
Speaker 2: Before the recent like in shitification wave.

683
00:31:34,760 --> 00:31:37,800
Speaker 4: Those I'm assuming I can swear on this, Yeah, yeah,

684
00:31:41,280 --> 00:31:46,640
Speaker 4: those organizations have historically been fairly more engineering driven, which

685
00:31:46,760 --> 00:31:51,080
Speaker 4: means that you typically have people higher in the organization

686
00:31:51,400 --> 00:31:55,640
Speaker 4: that are technical and have been programmers and who understand

687
00:31:55,680 --> 00:31:58,680
Speaker 4: some of the implications, and so they tend to try

688
00:31:58,760 --> 00:32:01,600
Speaker 4: at least we try to run interference with management and

689
00:32:01,720 --> 00:32:03,920
Speaker 4: to try to, you know, make sure everybody's on the

690
00:32:03,920 --> 00:32:06,680
Speaker 4: same page and that kind of stuff. A lot of

691
00:32:08,040 --> 00:32:09,760
Speaker 4: a lot, not all, but a lot of problems can

692
00:32:09,840 --> 00:32:13,960
Speaker 4: get get lessened if you have people in the organization

693
00:32:14,040 --> 00:32:16,160
Speaker 4: that are at higher level whose job is not to

694
00:32:16,320 --> 00:32:18,960
Speaker 4: manage people, but whose job is basically to keep track

695
00:32:19,040 --> 00:32:21,640
Speaker 4: and coordinate between different groups that are doing different technical

696
00:32:21,720 --> 00:32:22,520
Speaker 4: things right, to.

697
00:32:22,520 --> 00:32:24,680
Speaker 1: Make sure people aren't building the same thing I'm guessing,

698
00:32:24,880 --> 00:32:26,960
Speaker 1: or are building the right thing in the right way.

699
00:32:27,320 --> 00:32:31,480
Speaker 4: Yeah, and that how what this group is building is

700
00:32:31,560 --> 00:32:33,440
Speaker 4: going to impact what this group is building at some

701
00:32:33,520 --> 00:32:35,560
Speaker 4: point in the future. And making sure that when you

702
00:32:35,640 --> 00:32:37,040
Speaker 4: get to the point where those two things need to

703
00:32:37,040 --> 00:32:39,000
Speaker 4: talk to each other, they're both aware enough of what

704
00:32:39,080 --> 00:32:40,680
Speaker 4: the other one is doing that the two things hook

705
00:32:40,720 --> 00:32:41,440
Speaker 4: together correctly.

706
00:32:42,720 --> 00:32:46,520
Speaker 1: Yes, So, based on my analysis at these companies, that's

707
00:32:46,600 --> 00:32:51,160
Speaker 1: definitely gone out the window. I mean, even with LLLM integrations.

708
00:32:51,200 --> 00:32:53,280
Speaker 1: So there was a Johnson and Johnson story that went

709
00:32:53,280 --> 00:32:55,040
Speaker 1: out Wall Street General a couple of weeks ago where

710
00:32:55,040 --> 00:32:58,120
Speaker 1: it was like they had eight hundred and ninety LM

711
00:32:58,240 --> 00:33:02,640
Speaker 1: project Generative AI project, of which Taitla the Pereto principle

712
00:33:02,680 --> 00:33:05,200
Speaker 1: wins again ten fifteen percent of them were actually useful.

713
00:33:05,800 --> 00:33:07,680
Speaker 1: And the thing that stunned me about that of them,

714
00:33:07,680 --> 00:33:10,840
Speaker 1: the fact to confirmed my biases, which I love, was

715
00:33:10,920 --> 00:33:13,520
Speaker 1: the fact that they were eight hundred and ninety at

716
00:33:13,520 --> 00:33:16,400
Speaker 1: the fucking things and no one was like should we

717
00:33:16,560 --> 00:33:20,400
Speaker 1: have this many that There was no like like selfare

718
00:33:20,440 --> 00:33:22,960
Speaker 1: engineering culture that was like, hey, are we all chasing

719
00:33:23,040 --> 00:33:25,640
Speaker 1: our tails? Is this useless? But it sounds like they

720
00:33:25,680 --> 00:33:27,840
Speaker 1: were all focused on their little boxes.

721
00:33:28,160 --> 00:33:29,560
Speaker 2: Yeah, I mean so the other thing.

722
00:33:29,760 --> 00:33:34,400
Speaker 4: So understand that again greatly oversimplifying a lot of the

723
00:33:34,560 --> 00:33:39,080
Speaker 4: new stuff that's happened with large language machines, large language models,

724
00:33:39,240 --> 00:33:44,640
Speaker 4: and generative AI. People didn't expect, right. It was kind

725
00:33:44,680 --> 00:33:47,120
Speaker 4: of a surprise when you throw a whole bunch more

726
00:33:47,200 --> 00:33:49,320
Speaker 4: data at a large language model and it started spitting

727
00:33:49,360 --> 00:33:52,600
Speaker 4: out text in a way that nobody really There was

728
00:33:52,680 --> 00:33:55,480
Speaker 4: no like mathematical reason to expect it to be able

729
00:33:55,520 --> 00:33:58,360
Speaker 4: to be as good at generating rottocomplete.

730
00:33:57,760 --> 00:33:58,360
Speaker 2: Stuff as it is.

731
00:33:58,440 --> 00:34:02,880
Speaker 4: It was, right, And so there's this belief that if

732
00:34:03,040 --> 00:34:06,200
Speaker 4: we did the thing and we unexpectedly got more than

733
00:34:06,280 --> 00:34:08,279
Speaker 4: we asked for, if we do more of the thing,

734
00:34:08,400 --> 00:34:11,000
Speaker 4: maybe we'll unexpectedly get more of what we wanted, right

735
00:34:11,640 --> 00:34:14,680
Speaker 4: that hasn't seemed to really pan out the last couple

736
00:34:14,719 --> 00:34:19,239
Speaker 4: of years from what I can see, but that we

737
00:34:19,400 --> 00:34:21,640
Speaker 4: don't really understand enough about this to know whether it's

738
00:34:21,680 --> 00:34:23,600
Speaker 4: going to work, So we might as well throw spaghetti

739
00:34:23,600 --> 00:34:25,279
Speaker 4: at the wall and see if it sticks, because it might.

740
00:34:26,640 --> 00:34:29,719
Speaker 4: Kind of mentality is kind of pervasive at the moment,

741
00:34:30,520 --> 00:34:32,719
Speaker 4: and everybody's there's a lot of fomo. There's a lot

742
00:34:32,800 --> 00:34:35,400
Speaker 4: of like, you know, well, our competitors are probably doing this,

743
00:34:35,600 --> 00:34:38,880
Speaker 4: and so we don't want to get left behind. It

744
00:34:39,080 --> 00:34:42,000
Speaker 4: kind of reminds me of the rumors that they talked

745
00:34:42,000 --> 00:34:44,120
Speaker 4: about back in the eighties when the CIA was doing

746
00:34:44,160 --> 00:34:46,880
Speaker 4: all this psychic research, because supposedly the Russians were doing

747
00:34:46,920 --> 00:34:49,640
Speaker 4: psychic research and it was all complete crap, but both

748
00:34:49,680 --> 00:34:52,200
Speaker 4: sides were convinced that the other side was making some progress,

749
00:34:52,200 --> 00:34:54,520
Speaker 4: and so everybody was dumping a ton of money into it.

750
00:34:55,080 --> 00:35:14,799
Speaker 1: LMM Kultrum exactly. Yes, the title of the episode, So, okay,

751
00:35:15,440 --> 00:35:20,040
Speaker 1: koltr aside, is this something you're seeing in software development though,

752
00:35:20,400 --> 00:35:22,400
Speaker 1: because I know I've seen that in management or it's

753
00:35:22,400 --> 00:35:23,759
Speaker 1: just going to like shut the ship in there. This

754
00:35:23,880 --> 00:35:26,320
Speaker 1: seems like it's an important thing, right or is this

755
00:35:26,480 --> 00:35:28,080
Speaker 1: Are you seeing it within software development?

756
00:35:29,080 --> 00:35:30,959
Speaker 2: So I am seeing it within.

757
00:35:32,280 --> 00:35:36,879
Speaker 4: Software planning, right, So when managers are sitting down and saying, Okay,

758
00:35:36,920 --> 00:35:38,400
Speaker 4: we need to build this new thing, we need to

759
00:35:38,440 --> 00:35:40,439
Speaker 4: create a new group, we need to split this group apart,

760
00:35:40,640 --> 00:35:42,120
Speaker 4: we need to decide what our headcount is going to

761
00:35:42,160 --> 00:35:44,800
Speaker 4: be for next year, there's a lot of okay, and

762
00:35:45,000 --> 00:35:46,320
Speaker 4: what do we think the AI is going to do

763
00:35:46,400 --> 00:35:46,719
Speaker 4: next year?

764
00:35:46,760 --> 00:35:48,160
Speaker 2: And how many headcount do we think that's going to

765
00:35:48,200 --> 00:35:48,520
Speaker 2: save us?

766
00:35:48,520 --> 00:35:50,880
Speaker 4: In that kind of thing, right, There are some companies

767
00:35:51,239 --> 00:35:56,879
Speaker 4: do a Lingo is one, Klarna is one OP sorry, BP,

768
00:35:57,400 --> 00:36:00,680
Speaker 4: the former British Petroleum of what last year had a

769
00:36:00,719 --> 00:36:02,520
Speaker 4: thing where they said they were cutting seventy percent of

770
00:36:02,560 --> 00:36:04,440
Speaker 4: their contract software developers.

771
00:36:04,480 --> 00:36:06,640
Speaker 1: And in most of these they've kind of rolled them

772
00:36:06,680 --> 00:36:07,279
Speaker 1: back as well.

773
00:36:07,680 --> 00:36:09,480
Speaker 2: And I don't think dual lingo has yet.

774
00:36:10,760 --> 00:36:12,640
Speaker 1: This is just being unfit to you. They like a

775
00:36:12,760 --> 00:36:15,839
Speaker 1: day ago, really just like that would kind of It's

776
00:36:15,880 --> 00:36:18,320
Speaker 1: so funny. It's so funny. It's so funny that this

777
00:36:18,480 --> 00:36:20,680
Speaker 1: just keeps happening in exactly the same way. It's like, oh,

778
00:36:20,719 --> 00:36:23,680
Speaker 1: what a surprise, human beings to do stuff. Yeah, but

779
00:36:23,800 --> 00:36:25,239
Speaker 1: it kind of gets back to I think what you've

780
00:36:25,239 --> 00:36:27,759
Speaker 1: said about everything with l lams. It's like you can

781
00:36:27,880 --> 00:36:30,719
Speaker 1: teach something to say, Yeah, I think the right The

782
00:36:30,760 --> 00:36:32,359
Speaker 1: thing you're looking for is this, but you can't teach

783
00:36:32,400 --> 00:36:34,480
Speaker 1: it context. And that's been a point you've made again

784
00:36:34,480 --> 00:36:37,400
Speaker 1: and again, Like it seems the job of a software

785
00:36:37,440 --> 00:36:41,439
Speaker 1: engineer is highly contextual, unless you're like in the earlier days.

786
00:36:41,840 --> 00:36:43,080
Speaker 2: Yeah, and I like it.

787
00:36:43,120 --> 00:36:45,960
Speaker 4: It sometimes to the Memento guy from the Momento movie, right,

788
00:36:46,040 --> 00:36:48,640
Speaker 4: where like can't form long term memories. Then do you

789
00:36:48,800 --> 00:36:50,360
Speaker 4: do you really want the Momento guy to be the

790
00:36:50,400 --> 00:36:53,520
Speaker 4: person that's building the software that makes the seven thirty

791
00:36:53,560 --> 00:36:59,480
Speaker 4: seven max be able to compensate for its control input. Yeah.

792
00:37:00,280 --> 00:37:03,719
Speaker 1: Well, the thing is, though, with that argument, they would argue,

793
00:37:03,760 --> 00:37:05,799
Speaker 1: and I know that there is a better argument here.

794
00:37:05,880 --> 00:37:07,640
Speaker 1: They would argue, well, what if we just give it

795
00:37:07,719 --> 00:37:10,600
Speaker 1: everything that's ever happened? What if we just show every

796
00:37:10,680 --> 00:37:13,400
Speaker 1: single thing we've ever done in GitHub? Surely then it

797
00:37:13,400 --> 00:37:14,240
Speaker 1: would understand.

798
00:37:16,360 --> 00:37:20,640
Speaker 4: So the what I have seen from the papers that

799
00:37:20,719 --> 00:37:25,720
Speaker 4: I have read is that lllms have a basically squishy

800
00:37:25,760 --> 00:37:28,640
Speaker 4: middle context problem kind of the way that you do right. So,

801
00:37:28,760 --> 00:37:31,680
Speaker 4: if somebody gives you a big document to read or

802
00:37:31,719 --> 00:37:33,680
Speaker 4: a big long documentary to watch or something, and then

803
00:37:33,680 --> 00:37:35,480
Speaker 4: they ask you questions. What they're going to find is

804
00:37:35,560 --> 00:37:37,520
Speaker 4: that you remember a lot more from the beginning of

805
00:37:37,560 --> 00:37:39,040
Speaker 4: that and the end of that than you do from

806
00:37:39,080 --> 00:37:41,920
Speaker 4: the middle of that. Right, and lllms have the same

807
00:37:42,000 --> 00:37:43,640
Speaker 4: kind of problem. Right, And the other problem that the

808
00:37:43,760 --> 00:37:46,759
Speaker 4: LMS seem to have is that when you give them

809
00:37:46,880 --> 00:37:49,920
Speaker 4: a whole bunch of instructions, just instructions, polled on instructions,

810
00:37:49,960 --> 00:37:54,760
Speaker 4: pulled on instructions, they can either get confused and forget

811
00:37:54,840 --> 00:37:57,560
Speaker 4: some of the instructions, or they deadlock, or they just

812
00:37:57,880 --> 00:38:00,160
Speaker 4: start going, Okay, I can't satisfy all of these I'm

813
00:38:00,160 --> 00:38:01,759
Speaker 4: not even going to bother to satisfy any of them,

814
00:38:02,480 --> 00:38:05,840
Speaker 4: or they'll pick one or two. The fact that you

815
00:38:06,000 --> 00:38:10,680
Speaker 4: can take a million tokens and you can stick that

816
00:38:10,960 --> 00:38:16,080
Speaker 4: in the memory block that the the GPU is going

817
00:38:16,120 --> 00:38:21,480
Speaker 4: to process, doesn't necessarily believe, doesn't necessarily mean that all

818
00:38:21,719 --> 00:38:24,120
Speaker 4: of the tokens in that memory block are actually going

819
00:38:24,200 --> 00:38:25,920
Speaker 4: to be treated equally and going.

820
00:38:25,840 --> 00:38:34,240
Speaker 2: To be understood. Right in theory, maybe if you could.

821
00:38:35,840 --> 00:38:40,399
Speaker 4: Train your if you could like custom train an LLM

822
00:38:41,120 --> 00:38:43,680
Speaker 4: and modify all of its weights based on exactly what

823
00:38:43,920 --> 00:38:46,560
Speaker 4: your stuff was, and do that like day after day

824
00:38:46,600 --> 00:38:50,160
Speaker 4: after day after day. As things changed, you would theoretically

825
00:38:50,280 --> 00:38:53,680
Speaker 4: get better. I still don't think it would be you know,

826
00:38:54,400 --> 00:38:57,400
Speaker 4: I still think would understand the context as well, but

827
00:38:57,520 --> 00:38:59,880
Speaker 4: that would be ridiculously expensive.

828
00:39:00,360 --> 00:39:05,480
Speaker 1: Yeah, and at that point you could train a person, yes.

829
00:39:05,480 --> 00:39:07,400
Speaker 4: I mean the person would probably be more annoying. So

830
00:39:07,920 --> 00:39:10,880
Speaker 4: that's I mean the point. I mean, a lot of

831
00:39:10,960 --> 00:39:14,200
Speaker 4: this is seems to be really you know, we don't

832
00:39:14,280 --> 00:39:16,799
Speaker 4: like dealing with the Prima Donna programmer kind of thing,

833
00:39:16,920 --> 00:39:22,480
Speaker 4: right that there's this you know, I mean not just programmers, right,

834
00:39:22,520 --> 00:39:25,000
Speaker 4: we don't also don't want to deal with the Prima

835
00:39:25,040 --> 00:39:28,080
Speaker 4: Donna reporters or the Prima Donna illustrators or just want

836
00:39:28,080 --> 00:39:28,480
Speaker 4: to get rid.

837
00:39:28,440 --> 00:39:31,760
Speaker 1: Of these people. Right. He's annoying. They ask for stuff,

838
00:39:31,800 --> 00:39:32,719
Speaker 1: they want money.

839
00:39:33,280 --> 00:39:37,719
Speaker 2: Yeah, and days off and sickly even you know, healthcare, and.

840
00:39:38,280 --> 00:39:42,960
Speaker 1: It's just disgusting, how Dad. It's so it's frustrating as

841
00:39:43,000 --> 00:39:47,279
Speaker 1: well because across software development and everything, but especially with

842
00:39:47,320 --> 00:39:50,840
Speaker 1: self A developers, it feels just very insulting because it

843
00:39:50,960 --> 00:39:53,839
Speaker 1: doesn't seem like this stuff. Actually, here's a better question.

844
00:39:54,560 --> 00:39:58,239
Speaker 1: Have you seen much of an improvement with like one

845
00:39:58,280 --> 00:40:00,200
Speaker 1: to oh three like these reasoning Do you I think

846
00:40:00,440 --> 00:40:03,880
Speaker 1: the reasoning models change things for the beta. If so, how.

847
00:40:05,239 --> 00:40:12,120
Speaker 2: So a little that they don't make as many stupid mistakes.

848
00:40:13,239 --> 00:40:15,399
Speaker 4: It is basically what it what what it boils down

849
00:40:15,440 --> 00:40:18,400
Speaker 4: to going back to your your first thing, though, right,

850
00:40:18,520 --> 00:40:21,359
Speaker 4: I mean so. There was a piece, actually a couple

851
00:40:21,360 --> 00:40:24,360
Speaker 4: of pieces recently. One of them was about, you know,

852
00:40:24,440 --> 00:40:26,440
Speaker 4: tech workers are just like the rest of us, They're miserable.

853
00:40:27,040 --> 00:40:29,239
Speaker 4: There I'll I'll give you blinks to these. The other

854
00:40:29,320 --> 00:40:32,440
Speaker 4: one was a Corey doctor opiece that was like the

855
00:40:32,760 --> 00:40:37,000
Speaker 4: future of Amazon coders is the present of Amazon warehouse

856
00:40:37,040 --> 00:40:42,160
Speaker 4: workers or vice versa. There's there's a lot of there

857
00:40:42,200 --> 00:40:46,520
Speaker 4: has been a lot of deference given to software developers

858
00:40:47,680 --> 00:40:51,560
Speaker 4: over time, because you know, we have been kind of

859
00:40:51,640 --> 00:40:54,439
Speaker 4: the engine that's made a lot of the last twenty

860
00:40:54,480 --> 00:40:58,640
Speaker 4: thirty years work, and there's a desire to make that

861
00:40:58,960 --> 00:41:01,680
Speaker 4: not so anymore, and to make us just as interchangeable

862
00:41:01,719 --> 00:41:06,399
Speaker 4: as everybody else. I guess, you know, from a from

863
00:41:06,440 --> 00:41:09,080
Speaker 4: a economic standpoint, I kind of don't blame them.

864
00:41:09,160 --> 00:41:10,960
Speaker 2: I understand why they're trying to do what they're doing.

865
00:41:11,040 --> 00:41:13,440
Speaker 4: I don't I mean, I don't think that the warehouse

866
00:41:13,440 --> 00:41:15,520
Speaker 4: workers should be treated the way the warehouse workers are treated,

867
00:41:15,920 --> 00:41:17,920
Speaker 4: you know, much less everybody else gets treated that way.

868
00:41:19,080 --> 00:41:23,920
Speaker 4: And it's been a lot worse since the giant layoffs

869
00:41:24,000 --> 00:41:28,799
Speaker 4: at Twitter now X. When that happened and the thing

870
00:41:28,960 --> 00:41:33,640
Speaker 4: didn't crash and completely burn like everybody was or not everybody,

871
00:41:33,680 --> 00:41:36,520
Speaker 4: but a lot of people were expecting it to, the

872
00:41:37,320 --> 00:41:41,720
Speaker 4: the sentiment became, well, maybe all this, all these software

873
00:41:41,719 --> 00:41:44,400
Speaker 4: developers aren't as important as they you know, we've always

874
00:41:44,400 --> 00:41:44,920
Speaker 4: thought they were.

875
00:41:46,040 --> 00:41:49,600
Speaker 2: And you know, we will see over time what the

876
00:41:50,000 --> 00:41:52,560
Speaker 2: end result of that is. My guess is it's going

877
00:41:52,640 --> 00:41:53,600
Speaker 2: to be end up being a mess.

878
00:41:54,360 --> 00:41:57,880
Speaker 4: But you know, I'm a I'm a software developer, right,

879
00:41:57,920 --> 00:42:01,400
Speaker 4: I'm gonna it behooves me for it to be a mess, right,

880
00:42:01,560 --> 00:42:04,080
Speaker 4: So it might just be my bias that's getting in

881
00:42:04,120 --> 00:42:04,399
Speaker 4: the way.

882
00:42:04,719 --> 00:42:06,920
Speaker 1: I actually I think that you're right though, because I

883
00:42:07,000 --> 00:42:10,359
Speaker 1: remember back in twenty twenty one and onward the kind

884
00:42:10,400 --> 00:42:13,759
Speaker 1: of post remote work, the remote works. There was the

885
00:42:13,800 --> 00:42:16,640
Speaker 1: whole anti remote work push, but there was the whole

886
00:42:16,719 --> 00:42:19,080
Speaker 1: quiet quitting and things like that. That's twenty twenty two

887
00:42:19,160 --> 00:42:22,640
Speaker 1: where it's like software engineering, they just they expect to

888
00:42:22,680 --> 00:42:24,960
Speaker 1: be treated so well because twenty twenty one's all the

889
00:42:25,080 --> 00:42:28,760
Speaker 1: insane hiring, right. You saw tech companies like parking software workers.

890
00:42:29,239 --> 00:42:31,319
Speaker 1: I think that played into it as well, where all

891
00:42:31,400 --> 00:42:34,160
Speaker 1: of these companies who chose to pay these software engineers,

892
00:42:34,200 --> 00:42:36,680
Speaker 1: they were the ones that made the offers, got pissed

893
00:42:36,719 --> 00:42:39,680
Speaker 1: off that they'd done. Someone thought we should cut all

894
00:42:39,800 --> 00:42:44,360
Speaker 1: labor down to size, and then along comes coding. Almost

895
00:42:44,400 --> 00:42:46,560
Speaker 1: makes me wonder if most of these venture capitalists talking

896
00:42:46,560 --> 00:42:50,720
Speaker 1: about this don't know how to code themselves. Yeah, gotta wonder.

897
00:42:51,120 --> 00:42:54,960
Speaker 2: I don't know many that do. Yeah, I know some

898
00:42:55,120 --> 00:42:56,560
Speaker 2: that have at some point.

899
00:42:56,760 --> 00:43:01,279
Speaker 1: But the best thing it's at some point it's like

900
00:43:01,360 --> 00:43:05,000
Speaker 1: they're not part of modern software development culture, which I

901
00:43:05,080 --> 00:43:07,560
Speaker 1: know sounds kind of wanky, but I mean, just how

902
00:43:07,640 --> 00:43:11,520
Speaker 1: an organization builds software feels like something they should know.

903
00:43:11,920 --> 00:43:13,200
Speaker 1: But then again, they don't know how to build a

904
00:43:13,200 --> 00:43:15,879
Speaker 1: real organization ethos. Who the fuck? Yeah?

905
00:43:16,320 --> 00:43:18,360
Speaker 2: Well, I mean, honestly a lot of it.

906
00:43:20,000 --> 00:43:24,280
Speaker 4: I've been in organizations that VC's basically killed, right because

907
00:43:24,800 --> 00:43:27,800
Speaker 4: you know, we built a thing. That thing was, you know,

908
00:43:28,719 --> 00:43:32,880
Speaker 4: a reasonable business, But vcs don't want a reasonable business.

909
00:43:33,000 --> 00:43:34,759
Speaker 4: They want either one hundred ex return or they want

910
00:43:34,800 --> 00:43:38,200
Speaker 4: to tax write off, and they don't want anything in between, right, yeah,

911
00:43:38,600 --> 00:43:41,000
Speaker 4: So I mean what what they're looking for is really

912
00:43:41,080 --> 00:43:43,799
Speaker 4: I mean, they're not trying to run a regular business, right,

913
00:43:43,840 --> 00:43:46,600
Speaker 4: They're not trying to do the normal process. They're trying

914
00:43:46,640 --> 00:43:49,440
Speaker 4: to either you know, hit one out of the park

915
00:43:49,880 --> 00:43:53,480
Speaker 4: or throw it away and move on. And so they're

916
00:43:53,560 --> 00:43:55,640
Speaker 4: they're the rules for them are different because what they're

917
00:43:55,640 --> 00:43:57,480
Speaker 4: trying to accomplish is not what the rest of us.

918
00:43:57,400 --> 00:43:58,920
Speaker 2: Are trying to accomplish. As a general rule.

919
00:44:00,080 --> 00:44:02,440
Speaker 1: The theme of the fucking show, it's just like, it's

920
00:44:02,560 --> 00:44:04,680
Speaker 1: just like you have these people that don't code saying

921
00:44:04,719 --> 00:44:07,279
Speaker 1: how cod is should code, like Dario amat Day the

922
00:44:07,360 --> 00:44:09,719
Speaker 1: other day saying that this year we're going to have

923
00:44:09,800 --> 00:44:14,160
Speaker 1: the first one person software company with a billion dollars

924
00:44:14,400 --> 00:44:17,720
Speaker 1: revenue or something like that, and it's just I feel

925
00:44:17,760 --> 00:44:20,120
Speaker 1: like there are some people who should not be allowed

926
00:44:20,160 --> 00:44:23,800
Speaker 1: to speak as much sometimes, but it's just frustrating and insulting.

927
00:44:23,840 --> 00:44:25,960
Speaker 1: And it's but now that you've got me thinking about it,

928
00:44:26,080 --> 00:44:28,080
Speaker 1: it does feel like this is an attempt to reset

929
00:44:28,160 --> 00:44:31,279
Speaker 1: the labor market finally coming for software developers. And I

930
00:44:31,320 --> 00:44:33,040
Speaker 1: don't mean finally in a good way, right.

931
00:44:33,719 --> 00:44:36,000
Speaker 4: I mean, it feels like that being in the being

932
00:44:36,080 --> 00:44:39,319
Speaker 4: in that organ being in that industry at the moment,

933
00:44:39,400 --> 00:44:40,400
Speaker 4: it really feels like that.

934
00:44:40,960 --> 00:44:41,960
Speaker 2: Is it scary right now?

935
00:44:43,120 --> 00:44:44,080
Speaker 1: Is it scary right now?

936
00:44:45,360 --> 00:44:50,000
Speaker 4: Not for me because I'm old enough to be semi retired, right,

937
00:44:50,840 --> 00:44:53,400
Speaker 4: But I mean, I've been talking to a lot of folks.

938
00:44:54,040 --> 00:44:57,279
Speaker 4: I've been having a interviewing how much folks that are

939
00:44:57,440 --> 00:45:00,000
Speaker 4: that are listeners from my channel and kind of trying

940
00:45:00,120 --> 00:45:02,839
Speaker 4: to get a feel for what's going on. And I've

941
00:45:02,920 --> 00:45:05,520
Speaker 4: talked to folks that are you know, like I said that,

942
00:45:05,840 --> 00:45:07,160
Speaker 4: I talked to some folks that were like, you know,

943
00:45:07,200 --> 00:45:09,319
Speaker 4: I work for a big bank. They're cramming copilot, dinner

944
00:45:09,360 --> 00:45:12,120
Speaker 4: throat or eeveryoneted or not. I've talked to some folks

945
00:45:12,200 --> 00:45:13,919
Speaker 4: that are like, every time I sit down with my boss,

946
00:45:13,960 --> 00:45:15,440
Speaker 4: I'm thinking that, you know, this is going to be

947
00:45:15,480 --> 00:45:17,279
Speaker 4: the day that I'm going to find out that my

948
00:45:17,400 --> 00:45:19,360
Speaker 4: group is getting cut the way the other three groups

949
00:45:19,400 --> 00:45:21,560
Speaker 4: in the company is getting cut.

950
00:45:22,000 --> 00:45:22,880
Speaker 2: There's a lot of.

951
00:45:24,840 --> 00:45:29,640
Speaker 4: Artificial productivity requirement increases kind of thing, which is like

952
00:45:30,000 --> 00:45:34,240
Speaker 4: like one, just you know, we you know, we expect

953
00:45:34,280 --> 00:45:37,719
Speaker 4: more tickets closed per you know, two week period than

954
00:45:38,040 --> 00:45:39,960
Speaker 4: you know we've had before because we were giving you

955
00:45:40,040 --> 00:45:42,080
Speaker 4: this AI now, so you ought to be more productive

956
00:45:42,160 --> 00:45:42,719
Speaker 4: that kind of thing.

957
00:45:43,600 --> 00:45:46,480
Speaker 1: Would the ticket necessarily be something that you just write

958
00:45:46,520 --> 00:45:48,120
Speaker 1: toad for it more than just.

959
00:45:48,760 --> 00:45:51,640
Speaker 2: Well, so generally it's more than just that. But but

960
00:45:51,840 --> 00:45:53,320
Speaker 2: generally the ticket.

961
00:45:54,120 --> 00:45:56,120
Speaker 4: That's kind of the way that we track the work

962
00:45:56,200 --> 00:45:58,920
Speaker 4: that we do in a lot of organizations, right, And

963
00:45:59,120 --> 00:46:01,840
Speaker 4: some tickets are like, I'm building a new thing, and

964
00:46:01,920 --> 00:46:04,320
Speaker 4: those are kind of easier to predict. And some tickets

965
00:46:04,360 --> 00:46:07,160
Speaker 4: are this thing isn't behaving right, go figure out where

966
00:46:07,160 --> 00:46:09,080
Speaker 4: the bug is. And those are a lot harder to predict,

967
00:46:09,560 --> 00:46:12,919
Speaker 4: but they have these things. Agile has this thing called

968
00:46:12,920 --> 00:46:15,800
Speaker 4: a velocity graph where basically you see how many tickets

969
00:46:15,840 --> 00:46:18,400
Speaker 4: per person get closed over time, and people want to

970
00:46:18,400 --> 00:46:22,240
Speaker 4: see the slope of that line change because they're giving

971
00:46:22,280 --> 00:46:22,640
Speaker 4: you AI.

972
00:46:24,320 --> 00:46:26,320
Speaker 1: And I'm guessing the people telling you to change that

973
00:46:26,440 --> 00:46:27,640
Speaker 1: don't know what they're talking about.

974
00:46:28,200 --> 00:46:29,279
Speaker 2: That seems to be the case.

975
00:46:29,800 --> 00:46:35,080
Speaker 4: Great, so I mean the good news in theory, right,

976
00:46:35,200 --> 00:46:36,959
Speaker 4: I don't know to what extent this is going to happen,

977
00:46:37,040 --> 00:46:41,120
Speaker 4: but in theory, if they keep telling people, you know,

978
00:46:41,560 --> 00:46:43,600
Speaker 4: that slope of that line should be changing because you

979
00:46:43,680 --> 00:46:46,280
Speaker 4: have AI. Now, over time, if we see the slope

980
00:46:46,320 --> 00:46:50,840
Speaker 4: of that line not changing though, right, then theoretically it

981
00:46:50,920 --> 00:46:54,680
Speaker 4: will be proof that the AI is not providing the

982
00:46:54,760 --> 00:46:58,399
Speaker 4: return that people expect it. Well, you're not using it, right, Well, yes,

983
00:46:58,480 --> 00:46:59,920
Speaker 4: there's always that you're not prompting it, right.

984
00:47:00,040 --> 00:47:04,640
Speaker 1: That is that is basically what I am people. One

985
00:47:04,680 --> 00:47:06,560
Speaker 1: of the many reasons what you want is like, I

986
00:47:06,680 --> 00:47:09,239
Speaker 1: want to have people that actually code on to talk

987
00:47:09,280 --> 00:47:12,480
Speaker 1: about this stuff, because it's really easy as a layman

988
00:47:12,560 --> 00:47:15,680
Speaker 1: myself and for others to just be like, oh but

989
00:47:15,880 --> 00:47:18,879
Speaker 1: this does replace coding, right, and it does? It sounds

990
00:47:18,920 --> 00:47:20,440
Speaker 1: like it really doesn't.

991
00:47:20,520 --> 00:47:21,160
Speaker 2: Like it can help.

992
00:47:21,200 --> 00:47:24,000
Speaker 1: It can be like a force multiplier to an extent,

993
00:47:24,120 --> 00:47:27,680
Speaker 1: but even past the initial steps, it just isn't there.

994
00:47:28,200 --> 00:47:31,600
Speaker 4: Well, I mean, so the best analogy I've always found

995
00:47:31,800 --> 00:47:35,319
Speaker 4: to writing code is actually just writing, right. I Mean,

996
00:47:35,400 --> 00:47:37,520
Speaker 4: you can get chat GBT to spit out a few

997
00:47:37,560 --> 00:47:42,160
Speaker 4: paragraphs for you, right, but you know, you end up with,

998
00:47:42,480 --> 00:47:44,399
Speaker 4: you know, the legal briefs that have the story that's

999
00:47:44,440 --> 00:47:47,200
Speaker 4: made up or the you know, just things that aren't

1000
00:47:47,320 --> 00:47:49,759
Speaker 4: connected to reality or stuff that you know, when people

1001
00:47:49,800 --> 00:47:53,200
Speaker 4: read them, they're like, I mean, you you can tell

1002
00:47:53,200 --> 00:47:55,800
Speaker 4: the difference between AI slot generated you know, like the

1003
00:47:56,200 --> 00:48:01,000
Speaker 4: stupid the insert from the Chicago Sun, Yeah, yeah, the

1004
00:48:01,040 --> 00:48:04,200
Speaker 4: Philadelphia Inquirer. You know, all the books, all the things

1005
00:48:04,239 --> 00:48:06,279
Speaker 4: you can do this summer, right that like made up

1006
00:48:06,320 --> 00:48:07,840
Speaker 4: books and all that kind of I mean, like, but

1007
00:48:08,000 --> 00:48:11,279
Speaker 4: even even the articles that weren't the ones that we're

1008
00:48:11,320 --> 00:48:13,759
Speaker 4: making up stuff. You read the you know, this is

1009
00:48:13,800 --> 00:48:15,200
Speaker 4: what's going to be happening this summer. This is what

1010
00:48:15,239 --> 00:48:16,520
Speaker 4: the weather's going to be like or whatever. And you're

1011
00:48:16,560 --> 00:48:20,480
Speaker 4: reading and you're like this this there's no like insight here,

1012
00:48:20,600 --> 00:48:24,359
Speaker 4: there's no thought here, there's no you know, there's nothing

1013
00:48:24,440 --> 00:48:26,160
Speaker 4: in here that I get to the end of this.

1014
00:48:26,239 --> 00:48:28,080
Speaker 4: I've read the whole thing. I understand the whole thing,

1015
00:48:28,160 --> 00:48:30,240
Speaker 4: but I don't have anything I can walk away.

1016
00:48:30,040 --> 00:48:34,080
Speaker 1: With, right and I AI agents aren't coming along to

1017
00:48:34,200 --> 00:48:37,040
Speaker 1: replace software. But that you're not scared of Devin.

1018
00:48:37,719 --> 00:48:42,080
Speaker 4: I am not scared of Devon, so I well, actually

1019
00:48:42,280 --> 00:48:44,160
Speaker 4: I kind of am. I am scared that Devon is

1020
00:48:44,200 --> 00:48:46,000
Speaker 4: going to make a mess of things and then more

1021
00:48:46,040 --> 00:48:47,680
Speaker 4: things are going to get hacked, and that's going to

1022
00:48:47,760 --> 00:48:49,279
Speaker 4: end up being worse for everybody.

1023
00:48:48,960 --> 00:48:50,360
Speaker 1: On the unit. Right, how would it do that?

1024
00:48:51,360 --> 00:48:53,200
Speaker 4: By I mean like we were talking about before, right,

1025
00:48:53,320 --> 00:48:56,400
Speaker 4: So when you write code that isn't secure, right, and

1026
00:48:56,520 --> 00:48:59,279
Speaker 4: you write code that you know uses a library that's

1027
00:48:59,320 --> 00:49:01,279
Speaker 4: got an old version of a thing that they that

1028
00:49:01,360 --> 00:49:03,160
Speaker 4: there's a known bug in it, but you don't bother

1029
00:49:03,280 --> 00:49:05,719
Speaker 4: to check to see if there's a fix for that bug.

1030
00:49:06,239 --> 00:49:09,000
Speaker 4: Or you don't use best practices when it comes to

1031
00:49:09,040 --> 00:49:12,080
Speaker 4: writing code and that kind of thing, or you don't

1032
00:49:12,120 --> 00:49:15,799
Speaker 4: think about the the kinds of maintainability issues that you're

1033
00:49:15,840 --> 00:49:18,279
Speaker 4: going to have, and you do things like you ship

1034
00:49:18,360 --> 00:49:21,359
Speaker 4: out code in a in an Internet of things thing

1035
00:49:21,400 --> 00:49:27,560
Speaker 4: a light bulb right, or Internet Wi Fi router that

1036
00:49:27,760 --> 00:49:30,279
Speaker 4: cannot be patched over the Internet that has a bug

1037
00:49:30,360 --> 00:49:32,960
Speaker 4: in it, right, And now it's like that thing is

1038
00:49:33,000 --> 00:49:35,480
Speaker 4: going to have a bug in it forever, and you're

1039
00:49:35,480 --> 00:49:37,440
Speaker 4: gonna have to find all the ones on the on

1040
00:49:37,560 --> 00:49:40,880
Speaker 4: the earth and turn them off before someone's not going

1041
00:49:40,920 --> 00:49:42,040
Speaker 4: to be able to take them and be able to

1042
00:49:42,080 --> 00:49:44,080
Speaker 4: hack them and use them to attack somebody else from there.

1043
00:49:44,719 --> 00:49:47,000
Speaker 1: I mean, IoT is a huge probe low. Oh yeah,

1044
00:49:47,080 --> 00:49:50,160
Speaker 1: but the cheap ones have like the spywab stuff and

1045
00:49:50,640 --> 00:49:52,319
Speaker 1: panto mining it.

1046
00:49:52,480 --> 00:49:54,719
Speaker 4: Just but yeah, the ones, the ones that have they

1047
00:49:54,760 --> 00:49:57,279
Speaker 4: have like really nasty vulnerabilities, and they have no way

1048
00:49:57,320 --> 00:50:01,000
Speaker 4: of being updated once they leave the factory, right, and

1049
00:50:01,080 --> 00:50:03,200
Speaker 4: it's just as long as they're out there, they're going

1050
00:50:03,239 --> 00:50:05,360
Speaker 4: to be a problem literally for everybody on the internet.

1051
00:50:06,080 --> 00:50:11,200
Speaker 1: Jesus, Well, what can to wrap us up? What can

1052
00:50:11,480 --> 00:50:14,759
Speaker 1: a new engineer, someone new to software development? What can

1053
00:50:14,840 --> 00:50:16,839
Speaker 1: they learn right now? You've kind of done a video

1054
00:50:16,880 --> 00:50:18,000
Speaker 1: on this, but I think it's a good place to

1055
00:50:18,040 --> 00:50:20,920
Speaker 1: wrap us up. What can they start learning to actually

1056
00:50:20,960 --> 00:50:22,640
Speaker 1: get ahead, to actually prepare for all of this.

1057
00:50:23,440 --> 00:50:26,800
Speaker 4: That's a really good question. So you can't these days,

1058
00:50:27,920 --> 00:50:31,920
Speaker 4: you can't really be able to be an engineer. You

1059
00:50:31,960 --> 00:50:36,200
Speaker 4: can't get hired as an engineer without some ability to

1060
00:50:36,320 --> 00:50:40,520
Speaker 4: talk about being able to do prompts and use you know,

1061
00:50:40,719 --> 00:50:42,680
Speaker 4: some kind of AI code editor or that kind of thing.

1062
00:50:43,200 --> 00:50:45,200
Speaker 4: It's just an expectation of the job. Now, whether it

1063
00:50:45,239 --> 00:50:48,920
Speaker 4: should be or not a different thing. The I mean,

1064
00:50:49,120 --> 00:50:52,560
Speaker 4: like I said before, there are situations where you tell

1065
00:50:52,600 --> 00:50:54,200
Speaker 4: it what you want and it will type faster than

1066
00:50:54,200 --> 00:50:57,200
Speaker 4: you possibly can. So you know that's not necessarily bad.

1067
00:50:57,239 --> 00:51:01,440
Speaker 4: You need to understand that you need to figure out Well, okay,

1068
00:51:01,760 --> 00:51:03,960
Speaker 4: I'll get back to something else. You need to figure

1069
00:51:03,960 --> 00:51:07,359
Speaker 4: out basically how to test the thing, right, So how

1070
00:51:07,400 --> 00:51:09,240
Speaker 4: do you make sure that the code that it spits

1071
00:51:09,280 --> 00:51:12,520
Speaker 4: out does what you meant it to do. And what

1072
00:51:12,600 --> 00:51:14,600
Speaker 4: I'm expecting is that we're going to spend more time

1073
00:51:14,920 --> 00:51:18,319
Speaker 4: thinking about testing and thinking more about, you know, trying

1074
00:51:18,360 --> 00:51:20,279
Speaker 4: to find exceptions and that kind of thing than we

1075
00:51:20,400 --> 00:51:22,520
Speaker 4: have in the past, because the code that's actually being

1076
00:51:22,600 --> 00:51:24,440
Speaker 4: generated is going to be less likely to be quality

1077
00:51:24,480 --> 00:51:28,440
Speaker 4: than it was in the past. Right. The problem is

1078
00:51:29,680 --> 00:51:32,320
Speaker 4: it has become the case in the in the programming

1079
00:51:32,360 --> 00:51:34,799
Speaker 4: industry that the things you need to do to get

1080
00:51:34,840 --> 00:51:37,240
Speaker 4: through the interview to get hired have very little resemblance

1081
00:51:37,280 --> 00:51:38,759
Speaker 4: to the things that you actually do on the job

1082
00:51:38,840 --> 00:51:41,239
Speaker 4: that you need to actually do a good job. And

1083
00:51:41,360 --> 00:51:44,279
Speaker 4: so that's a whole different We could probably have a

1084
00:51:44,360 --> 00:51:47,360
Speaker 4: whole other podcast episode just about the interviewing problem.

1085
00:51:48,280 --> 00:51:53,759
Speaker 2: But the main thing right now, it's so right now

1086
00:51:53,920 --> 00:51:55,120
Speaker 2: the whole hiring thing.

1087
00:51:55,200 --> 00:51:57,520
Speaker 4: And this isn't I don't think true for just programmers,

1088
00:51:57,560 --> 00:52:01,400
Speaker 4: but it's especially true for programmers. Is all you know,

1089
00:52:02,280 --> 00:52:05,520
Speaker 4: bots that customize your resume and write a custom color

1090
00:52:05,640 --> 00:52:08,880
Speaker 4: cover letter and then send them over to the submit

1091
00:52:08,960 --> 00:52:11,520
Speaker 4: the thing to the bot that's screening the resume and

1092
00:52:11,600 --> 00:52:14,600
Speaker 4: screening right, and that getting it to the point where

1093
00:52:14,600 --> 00:52:16,080
Speaker 4: you can actually talk to a human is a nightmare

1094
00:52:16,120 --> 00:52:18,000
Speaker 4: right now, So the whole hiring system is kind of broken,

1095
00:52:19,920 --> 00:52:21,759
Speaker 4: so that the actually getting to the point where you

1096
00:52:21,800 --> 00:52:24,759
Speaker 4: can get hired is a nightmare at the moment. But

1097
00:52:25,400 --> 00:52:29,320
Speaker 4: the thing that you can do is figure out what

1098
00:52:29,640 --> 00:52:32,399
Speaker 4: kinds of things that AI are good at is good at.

1099
00:52:33,280 --> 00:52:34,759
Speaker 4: And one of the things that AI is pretty good

1100
00:52:34,760 --> 00:52:37,680
Speaker 4: at is things that don't matter as much, right. So,

1101
00:52:37,920 --> 00:52:41,200
Speaker 4: like you know, AI can pick the layout of a

1102
00:52:41,280 --> 00:52:43,319
Speaker 4: site potentially right, and you could have it picked two

1103
00:52:43,400 --> 00:52:45,759
Speaker 4: or three of them, and you can basically do what's

1104
00:52:45,800 --> 00:52:47,880
Speaker 4: called an A B test, and you can randomly assign

1105
00:52:47,920 --> 00:52:49,200
Speaker 4: people to it. You can figure out which one of

1106
00:52:49,200 --> 00:52:50,880
Speaker 4: them performs better, and you can throw the rest.

1107
00:52:50,760 --> 00:52:51,160
Speaker 2: Of the money.

1108
00:52:51,320 --> 00:52:53,359
Speaker 1: And even then at some point you will probably want

1109
00:52:53,400 --> 00:52:54,560
Speaker 1: the design customized.

1110
00:52:55,080 --> 00:52:56,160
Speaker 2: Yeah, I mean, but but.

1111
00:52:58,640 --> 00:53:00,799
Speaker 4: I think there will be a lot of things where

1112
00:53:02,160 --> 00:53:05,839
Speaker 4: people can kind of get something that's kind of good

1113
00:53:05,960 --> 00:53:09,759
Speaker 4: enough to get started right, right. And I think that

1114
00:53:09,960 --> 00:53:11,719
Speaker 4: to some extent this is going to be kind of

1115
00:53:11,760 --> 00:53:14,319
Speaker 4: a boon for the industry in the longer term where

1116
00:53:14,520 --> 00:53:17,719
Speaker 4: somebody who can't program right now, but who has some

1117
00:53:17,840 --> 00:53:19,719
Speaker 4: idea of kind of what they want can do like

1118
00:53:19,800 --> 00:53:22,880
Speaker 4: a vibe coding thing. They can validate that the market

1119
00:53:23,040 --> 00:53:27,279
Speaker 4: that they want to try to attack exists, right, and

1120
00:53:27,360 --> 00:53:29,239
Speaker 4: that people want to use the kind of thing that

1121
00:53:29,320 --> 00:53:31,120
Speaker 4: they built, and then they can bring in somebody to

1122
00:53:31,160 --> 00:53:34,160
Speaker 4: actually build it, right, you know what I mean. And

1123
00:53:34,280 --> 00:53:36,360
Speaker 4: those kinds of things wouldn't necessarily have been able to

1124
00:53:36,480 --> 00:53:39,319
Speaker 4: happen in the complete absence of AI.

1125
00:53:39,400 --> 00:53:41,040
Speaker 2: So it's not, I don't think, completely useless.

1126
00:53:41,719 --> 00:53:44,840
Speaker 4: And there there's times when as a as a developer,

1127
00:53:45,000 --> 00:53:47,440
Speaker 4: there are things that we're not good at, like you know,

1128
00:53:47,520 --> 00:53:49,640
Speaker 4: writing marketing copy and that kind of stuff that if

1129
00:53:49,680 --> 00:53:51,680
Speaker 4: we're trying to do a project for ourselves, you know,

1130
00:53:51,760 --> 00:53:53,960
Speaker 4: a lot of that stuff we can just outsource to

1131
00:53:54,000 --> 00:53:56,680
Speaker 4: the AI because it's not the thing that keeps the

1132
00:53:57,200 --> 00:53:59,759
Speaker 4: project from actually breaking and getting hacked in that kind

1133
00:53:59,760 --> 00:54:02,040
Speaker 4: of thing, right. So it's kind of like there's this

1134
00:54:02,160 --> 00:54:04,839
Speaker 4: concept where you need to keep the things that are

1135
00:54:04,960 --> 00:54:07,759
Speaker 4: part of your competitive advantage in house, and everything else

1136
00:54:07,840 --> 00:54:10,080
Speaker 4: you can kind of outsource to somebody else. The kinds

1137
00:54:10,080 --> 00:54:11,719
Speaker 4: of things you can outsource to somebody else are the

1138
00:54:11,760 --> 00:54:13,239
Speaker 4: kinds of things that you potentially you could throw an

1139
00:54:13,239 --> 00:54:14,440
Speaker 4: AI at because they're.

1140
00:54:14,239 --> 00:54:17,400
Speaker 1: Not even even then it's like, it doesn't seem like

1141
00:54:17,480 --> 00:54:20,520
Speaker 1: that's a ton of things right now or will.

1142
00:54:20,440 --> 00:54:25,120
Speaker 2: Be again, it's the so it's basically two things.

1143
00:54:25,160 --> 00:54:27,840
Speaker 4: It's things that where the quality of the thing doesn't

1144
00:54:27,880 --> 00:54:32,120
Speaker 4: matter really, right, which every business has those kinds of things, right,

1145
00:54:32,400 --> 00:54:36,280
Speaker 4: And they're the kinds of things where you can define

1146
00:54:36,320 --> 00:54:39,520
Speaker 4: a metric that you can test the AI against and

1147
00:54:39,640 --> 00:54:41,080
Speaker 4: let it try over and over and over and over

1148
00:54:41,080 --> 00:54:42,440
Speaker 4: and over again until it gets to the point where

1149
00:54:42,440 --> 00:54:43,000
Speaker 4: it's good enough.

1150
00:54:43,960 --> 00:54:44,160
Speaker 1: Yeah.

1151
00:54:44,200 --> 00:54:47,200
Speaker 4: Right, So if your metric is more people click on

1152
00:54:47,320 --> 00:54:51,640
Speaker 4: this button than the button before, right, then you can

1153
00:54:51,719 --> 00:54:53,560
Speaker 4: have the AI create a whole bunch of different ways

1154
00:54:53,600 --> 00:54:56,719
Speaker 4: to skin that button, right, and then you can say, okay,

1155
00:54:56,840 --> 00:54:58,560
Speaker 4: so the one that tested best is the one we're

1156
00:54:58,560 --> 00:55:00,279
Speaker 4: going to keep. That's the thing you can throw an

1157
00:55:00,280 --> 00:55:03,000
Speaker 4: AI at, right, because you've got a well defined way

1158
00:55:03,040 --> 00:55:05,680
Speaker 4: of checking in no telling how long it's going to take,

1159
00:55:05,880 --> 00:55:07,360
Speaker 4: but you have a well defined way of checking to

1160
00:55:07,400 --> 00:55:08,520
Speaker 4: see if it's working right or not.

1161
00:55:09,480 --> 00:55:12,760
Speaker 1: So yeah, I mean, for years I've had the theory

1162
00:55:12,840 --> 00:55:15,920
Speaker 1: that this industry was a twenty to twenty five billion

1163
00:55:15,960 --> 00:55:19,520
Speaker 1: dollar total addressable market pretending to be a trillion dollar one.

1164
00:55:19,719 --> 00:55:21,759
Speaker 1: And everything you're saying really is just it's like you're

1165
00:55:21,800 --> 00:55:24,919
Speaker 1: describing things like platform as a service. They like like, yeah,

1166
00:55:25,160 --> 00:55:28,440
Speaker 1: things that you use in tandem with very real people

1167
00:55:28,480 --> 00:55:29,720
Speaker 1: in intentional ideas.

1168
00:55:31,200 --> 00:55:33,839
Speaker 4: Yeah, this is I don't see a world in which

1169
00:55:33,960 --> 00:55:37,160
Speaker 4: this is a we replace all the humans. You know

1170
00:55:37,560 --> 00:55:40,560
Speaker 4: that the whole Like, you know, this is going to

1171
00:55:40,640 --> 00:55:42,919
Speaker 4: displace eighty percent of the white color workers in the world.

1172
00:55:43,600 --> 00:55:47,839
Speaker 4: I just you know that the only the only people

1173
00:55:47,880 --> 00:55:50,759
Speaker 4: that are really going to be replaced anytime soon are

1174
00:55:50,840 --> 00:55:54,239
Speaker 4: people that either weren't doing a great job to start with,

1175
00:55:54,560 --> 00:55:59,080
Speaker 4: or people whose bosses don't understand what they were doing

1176
00:55:59,360 --> 00:56:01,560
Speaker 4: to the point the boss thought that what they were

1177
00:56:01,600 --> 00:56:04,200
Speaker 4: doing mattered. And my guess is that there's going to

1178
00:56:04,239 --> 00:56:06,319
Speaker 4: be regret at that point and that at some point

1179
00:56:06,360 --> 00:56:07,759
Speaker 4: they're gonna have to bring those people back.

1180
00:56:08,840 --> 00:56:12,480
Speaker 1: Well, Carle, this has been such a wonderful conversation. Where

1181
00:56:12,480 --> 00:56:13,239
Speaker 1: can people find you?

1182
00:56:14,040 --> 00:56:16,600
Speaker 4: I am Internet of Bugs at YouTube is probably the

1183
00:56:16,640 --> 00:56:19,440
Speaker 4: easiest place to find me, and then there are links

1184
00:56:19,560 --> 00:56:21,359
Speaker 4: on that channel to point at other things.

1185
00:56:21,880 --> 00:56:23,680
Speaker 1: And you've been listening to me at Zichron you've been

1186
00:56:23,680 --> 00:56:26,839
Speaker 1: listening to Better Offline. Thank you everyone for listening, and yeah,

1187
00:56:26,880 --> 00:56:27,759
Speaker 1: we'll catch you next week.

1188
00:56:35,920 --> 00:56:37,440
Speaker 2: Thank you for listening to Better Offline.

1189
00:56:37,719 --> 00:56:40,120
Speaker 1: The editor and composer of the Better Offline theme song

1190
00:56:40,200 --> 00:56:42,839
Speaker 1: is Mattawsowski. You can check out more of his music

1191
00:56:42,880 --> 00:56:46,360
Speaker 1: and audio projects at Mattasowski dot com, m A T

1192
00:56:46,520 --> 00:56:50,960
Speaker 1: T O S O W s ki dot com. You

1193
00:56:51,040 --> 00:56:53,439
Speaker 1: can email me at easy at Better offline dot com

1194
00:56:53,640 --> 00:56:55,920
Speaker 1: or visit better Offline dot com to find more podcast

1195
00:56:56,000 --> 00:56:59,320
Speaker 1: links and of course, my newsletter. I also really recommend

1196
00:56:59,320 --> 00:57:01,279
Speaker 1: you go to chat Where's your Head dot at to

1197
00:57:01,360 --> 00:57:04,160
Speaker 1: visit the discord, and go to our slash Better Offline

1198
00:57:04,200 --> 00:57:07,400
Speaker 1: to check out I'll Reddit. Thank you so much for listening.

1199
00:57:08,280 --> 00:57:10,920
Speaker 3: Better Offline is a production of cool Zone Media. For

1200
00:57:11,080 --> 00:57:14,680
Speaker 3: more from cool Zone Media, visit our website Coolzonemedia dot com,

1201
00:57:15,200 --> 00:57:18,040
Speaker 3: or check us out on the iHeartRadio app, Apple Podcasts,

1202
00:57:18,160 --> 00:57:19,600
Speaker 3: or wherever you get your podcasts