1
00:00:05,200 --> 00:00:10,000
Speaker 1: What is a brain computer interface? How far along is

2
00:00:10,160 --> 00:00:14,120
Speaker 1: this field? Can we evesdrop on the brain so that

3
00:00:14,160 --> 00:00:18,040
Speaker 1: a person who has lost the ability to move can

4
00:00:18,200 --> 00:00:22,120
Speaker 1: use their brain to control a computer cursor or a

5
00:00:22,239 --> 00:00:25,880
Speaker 1: robotic arm. Can someone who has lost the ability to

6
00:00:26,040 --> 00:00:30,760
Speaker 1: speak send brain signals to a decoder and hear their

7
00:00:30,880 --> 00:00:36,360
Speaker 1: voice again? Can we restore autonomy and dignity and eventually

8
00:00:36,800 --> 00:00:41,720
Speaker 1: do so so seamlessly that the technology disappears and the

9
00:00:41,760 --> 00:00:46,920
Speaker 1: person reappears In the future, where will the ethical boundaries

10
00:00:46,960 --> 00:00:52,320
Speaker 1: be between restoring function and spying on private thought? And

11
00:00:52,400 --> 00:00:57,400
Speaker 1: who owns the stream of neural data that represents you?

12
00:01:00,640 --> 00:01:03,880
Speaker 1: Welcome to Inner Cosmos with me David Eagleman. I'm a

13
00:01:03,920 --> 00:01:07,720
Speaker 1: neuroscientist and author at Stanford and in these episodes we

14
00:01:07,840 --> 00:01:12,560
Speaker 1: sail deeply into our three pound universe to understand why

15
00:01:12,680 --> 00:01:31,839
Speaker 1: and how our lives look the way they do. This week,

16
00:01:31,840 --> 00:01:36,760
Speaker 1: we're talking about technology for reading the brain. Now. One

17
00:01:36,760 --> 00:01:40,480
Speaker 1: thing that I find fascinating is that ancient cultures didn't

18
00:01:40,520 --> 00:01:44,160
Speaker 1: care at all about the brain. They generally would just

19
00:01:44,680 --> 00:01:48,720
Speaker 1: throw it out at autopsy, and it's understandable why it

20
00:01:48,880 --> 00:01:53,360
Speaker 1: just looks and feels like a huge, squishy walnut. If

21
00:01:53,360 --> 00:01:57,200
Speaker 1: you could sit and stare at a brain in action,

22
00:01:57,880 --> 00:02:03,200
Speaker 1: you wouldn't see anything happening. So it's taken centuries and

23
00:02:03,240 --> 00:02:06,160
Speaker 1: a lot of technology to realize that, in fact, the

24
00:02:06,200 --> 00:02:11,680
Speaker 1: brain is alive with lots of tiny cells, microscopically tiny,

25
00:02:12,040 --> 00:02:15,960
Speaker 1: and these cells are transmitting electrical signals tens or one

26
00:02:16,040 --> 00:02:18,920
Speaker 1: hundred times every second for each cell. And you have

27
00:02:19,080 --> 00:02:23,880
Speaker 1: eighty six billion of these cells. So this big, squishy

28
00:02:23,919 --> 00:02:27,799
Speaker 1: walnut is one of the busiest things on the planet.

29
00:02:28,680 --> 00:02:32,560
Speaker 1: But because it is so fragile, Mother Nature surrounds the

30
00:02:32,600 --> 00:02:36,839
Speaker 1: brain with an armored bunker plating the skull, and that

31
00:02:36,919 --> 00:02:40,080
Speaker 1: provides a huge challenge if you want to go in

32
00:02:40,120 --> 00:02:44,600
Speaker 1: there and eavesdrop on what the cells are doing. Now,

33
00:02:44,639 --> 00:02:47,400
Speaker 1: why would you want to spy on these cells? Well,

34
00:02:47,840 --> 00:02:52,800
Speaker 1: imagine if your thoughts could exit the skull as easily

35
00:02:53,160 --> 00:02:57,440
Speaker 1: as words leave your mouth. Now, there's a sense in

36
00:02:57,480 --> 00:03:00,840
Speaker 1: which we always do this. We use keyboards, touch screens,

37
00:03:00,919 --> 00:03:05,079
Speaker 1: and voice assistants, but all of those are detours. They

38
00:03:05,160 --> 00:03:09,160
Speaker 1: force the brain to root its intentions through muscle, and

39
00:03:09,240 --> 00:03:13,520
Speaker 1: that's fine if your muscles work. The problem is that

40
00:03:13,720 --> 00:03:17,680
Speaker 1: lots of people, millions of our neighbors and friends don't

41
00:03:17,680 --> 00:03:21,000
Speaker 1: have a way to get the information out of their

42
00:03:21,040 --> 00:03:24,959
Speaker 1: brain because something about the brain or the brain's pathways

43
00:03:25,080 --> 00:03:28,639
Speaker 1: or the muscles are not working, and therefore their brain

44
00:03:28,800 --> 00:03:31,640
Speaker 1: knows what they want to do or say, but there's

45
00:03:31,639 --> 00:03:35,280
Speaker 1: no way to get that information out. And this is

46
00:03:35,320 --> 00:03:39,800
Speaker 1: where the idea of a brain computer interface comes in.

47
00:03:40,160 --> 00:03:44,520
Speaker 1: What you'll hear referred to as a BCEI brain computer interface.

48
00:03:45,000 --> 00:03:48,360
Speaker 1: The idea of a BCI is to listen directly to

49
00:03:48,440 --> 00:03:52,320
Speaker 1: the neural patterns that mean move or speak or select,

50
00:03:52,680 --> 00:03:56,800
Speaker 1: and then you use some device to translate those patterns

51
00:03:56,880 --> 00:04:01,840
Speaker 1: directly into activation in the outside world. Now, as I said,

52
00:04:01,840 --> 00:04:04,160
Speaker 1: this is a huge deal for all the people for

53
00:04:04,200 --> 00:04:09,480
Speaker 1: whom the path from intention to movement has been interrupted

54
00:04:09,520 --> 00:04:12,920
Speaker 1: by disease or injury. The intent is still alive and

55
00:04:12,960 --> 00:04:16,800
Speaker 1: well in the cortex, and BCIs are the bridge back.

56
00:04:17,279 --> 00:04:22,239
Speaker 1: They turn silent plans into text or voice or cursor

57
00:04:22,320 --> 00:04:27,120
Speaker 1: control or reaching and grasping. But the story will, at

58
00:04:27,200 --> 00:04:30,839
Speaker 1: least in theory, reach beyond the medical because once you

59
00:04:30,920 --> 00:04:34,240
Speaker 1: can read out the programs for say this word or

60
00:04:34,320 --> 00:04:38,400
Speaker 1: press that key, now you've built a communication channel between

61
00:04:38,520 --> 00:04:43,919
Speaker 1: biological tissue and silicon, and that opens new forms of

62
00:04:44,080 --> 00:04:49,760
Speaker 1: interaction that our species has barely begun to imagine. Now,

63
00:04:49,839 --> 00:04:51,960
Speaker 1: let me not get ahead of myself yet, because as

64
00:04:52,000 --> 00:04:54,599
Speaker 1: we're going to see today, we are still at the

65
00:04:54,760 --> 00:04:58,240
Speaker 1: earliest stages of this technology. But this is what we're

66
00:04:58,279 --> 00:05:01,279
Speaker 1: going to talk about at the end. Now, you can

67
00:05:01,360 --> 00:05:05,160
Speaker 1: build bceiyes in lots of flavors. Some rest on the scalp,

68
00:05:05,480 --> 00:05:08,840
Speaker 1: Others sit on the surface of the brain. Others poke

69
00:05:09,160 --> 00:05:12,440
Speaker 1: tiny wires called electrodes into the surface of the brain

70
00:05:12,880 --> 00:05:15,480
Speaker 1: or even down deep into the brain for some purposes.

71
00:05:16,240 --> 00:05:20,520
Speaker 1: Some of these BCIs only read the electrical activity. Others

72
00:05:20,560 --> 00:05:25,120
Speaker 1: will also write with electrical patterns that the brain experiences

73
00:05:25,200 --> 00:05:28,200
Speaker 1: as touch or sound or sight. In every case, the

74
00:05:28,240 --> 00:05:32,360
Speaker 1: principle is the same. Brains issue commands, and they're very

75
00:05:32,520 --> 00:05:36,760
Speaker 1: fast and complex internal language of electrical spikes. This is

76
00:05:36,800 --> 00:05:41,000
Speaker 1: a language that we haven't nearly decoded yet, but machines

77
00:05:41,120 --> 00:05:44,159
Speaker 1: can learn to translate that language through a lot of

78
00:05:44,360 --> 00:05:48,839
Speaker 1: trial and error. Huge populations of neurons are playing some

79
00:05:49,240 --> 00:05:54,000
Speaker 1: symphony piece, and these decoders learn how to hear the

80
00:05:54,080 --> 00:05:58,240
Speaker 1: music and root the commands to a cursor or a

81
00:05:58,279 --> 00:06:02,040
Speaker 1: speaker or a robotic arm or whatever. Now. The issue

82
00:06:02,080 --> 00:06:04,120
Speaker 1: is that when we talk about it, it all seems

83
00:06:04,200 --> 00:06:08,240
Speaker 1: very straightforward and easy, but actually getting in there and

84
00:06:08,320 --> 00:06:12,320
Speaker 1: getting technology that can record from these microscopic little cells,

85
00:06:12,560 --> 00:06:16,520
Speaker 1: having these little changes in their electrical potential of tens

86
00:06:16,520 --> 00:06:20,640
Speaker 1: of millivolts, and making a system that lasts, and then

87
00:06:20,720 --> 00:06:23,960
Speaker 1: putting all the data together to understand what this very

88
00:06:24,120 --> 00:06:28,320
Speaker 1: tiny sampling of neurons, maybe a few hundred out of

89
00:06:28,640 --> 00:06:32,200
Speaker 1: hundreds of billions of neurons. It turns out this is

90
00:06:32,240 --> 00:06:37,640
Speaker 1: a massive engineering challenge and there are a million practical questions.

91
00:06:38,000 --> 00:06:41,640
Speaker 1: How reliable are these systems outside the lab? Can they

92
00:06:41,680 --> 00:06:46,480
Speaker 1: survive infection and signal drift? What about battery life? What's

93
00:06:46,560 --> 00:06:50,559
Speaker 1: the surgical risk? When does insurance cover these? So there's

94
00:06:50,800 --> 00:06:55,520
Speaker 1: a huge gap between a beautiful proof of principle and

95
00:06:55,800 --> 00:07:00,440
Speaker 1: a device that changes lives every day, and crossing that

96
00:07:00,560 --> 00:07:03,440
Speaker 1: gap is the real work of the field right now.

97
00:07:04,160 --> 00:07:06,320
Speaker 1: Now there's also a second issue. As soon as we

98
00:07:06,360 --> 00:07:10,840
Speaker 1: start talking about reading the brain, the questions start to surface,

99
00:07:11,000 --> 00:07:14,880
Speaker 1: what exactly are we reading? Is it intended movements? That's

100
00:07:14,920 --> 00:07:18,360
Speaker 1: one thing is that inner speech? Is it where you

101
00:07:18,520 --> 00:07:22,120
Speaker 1: place your attention? You can imagine situations in which there

102
00:07:22,160 --> 00:07:25,760
Speaker 1: are things that you don't want everyone knowing. We're used

103
00:07:25,760 --> 00:07:29,640
Speaker 1: to the skull having some sort of sanctity. So where

104
00:07:29,680 --> 00:07:36,080
Speaker 1: will the ethical boundaries be between restoring function and evesdropping

105
00:07:36,120 --> 00:07:40,040
Speaker 1: on private thought? Who's going to own the stream of

106
00:07:40,120 --> 00:07:44,320
Speaker 1: data that is literally you? How do we guarantee consent

107
00:07:44,440 --> 00:07:48,680
Speaker 1: and security and dignity when the interface is not on

108
00:07:48,720 --> 00:07:52,280
Speaker 1: your desk but inside your skull. So, even in the

109
00:07:52,280 --> 00:07:54,800
Speaker 1: face of all the tough questions coming down the pike,

110
00:07:55,320 --> 00:07:59,840
Speaker 1: it's hard not to feel awe at what's already possible.

111
00:07:59,840 --> 00:08:04,040
Speaker 1: Who have been locked inside their bodies are communicating again.

112
00:08:04,360 --> 00:08:07,080
Speaker 1: They're talking with their loved ones for the first time

113
00:08:07,160 --> 00:08:12,360
Speaker 1: in years. And the technology keeps improving every month, smarter algorithms,

114
00:08:12,440 --> 00:08:17,960
Speaker 1: better sensors, cleaner signals, and crucially designs that move from

115
00:08:17,960 --> 00:08:20,960
Speaker 1: the hospital to the home. So today I want to

116
00:08:21,000 --> 00:08:23,600
Speaker 1: explore what that looks like and where we are in

117
00:08:23,600 --> 00:08:26,480
Speaker 1: the process and where things are going. So I sat

118
00:08:26,520 --> 00:08:29,800
Speaker 1: down with my colleague Sergei Stavisky. Sergei is at the

119
00:08:29,840 --> 00:08:34,280
Speaker 1: UC Davis Neuroprosthetics Lab, which he co directs with neurosurgeon

120
00:08:34,480 --> 00:08:38,720
Speaker 1: David Brandman. With their collaborators, they work on BCIs that

121
00:08:38,840 --> 00:08:43,400
Speaker 1: restore communication and they're pushing towards systems that are fast

122
00:08:43,520 --> 00:08:47,600
Speaker 1: and expressive and practical for everyday life. So here's my

123
00:08:47,679 --> 00:08:50,280
Speaker 1: interview with Sergei Staviski.

124
00:08:53,920 --> 00:08:58,120
Speaker 2: A brain computer interface is a device that interacts between

125
00:08:58,200 --> 00:09:01,000
Speaker 2: technology and a brains. You have the brain, you have

126
00:09:01,240 --> 00:09:04,200
Speaker 2: some way of getting information in or out, and you

127
00:09:04,280 --> 00:09:07,560
Speaker 2: have some computation that's happening. And that computation it could

128
00:09:07,559 --> 00:09:09,240
Speaker 2: be happening inside the body, so it could be a

129
00:09:09,320 --> 00:09:12,240
Speaker 2: chip that does everything in the brain, or it could

130
00:09:12,240 --> 00:09:15,800
Speaker 2: be sending that information to a laptop next to the person,

131
00:09:15,880 --> 00:09:18,079
Speaker 2: or even to the cloud for more computation.

132
00:09:18,480 --> 00:09:21,080
Speaker 1: Now, one of your interests is that you know, over

133
00:09:21,120 --> 00:09:23,440
Speaker 1: a century ago people figured out you could dunk an

134
00:09:23,480 --> 00:09:27,480
Speaker 1: electrode into the brain the thin wire and because cells

135
00:09:27,480 --> 00:09:33,320
Speaker 1: are communicating with little electrical signals, you're you can eavesdrop

136
00:09:33,440 --> 00:09:36,440
Speaker 1: on that and you can also stimulate the cell to

137
00:09:36,480 --> 00:09:39,800
Speaker 1: do whatever. So tell us about the history of this,

138
00:09:41,080 --> 00:09:43,880
Speaker 1: how people have thought about, let's eavesdrop on the brain

139
00:09:43,960 --> 00:09:45,240
Speaker 1: and turn that into something.

140
00:09:45,480 --> 00:09:49,440
Speaker 2: So starting in the sixties and seventies and eighties, especially

141
00:09:49,480 --> 00:09:52,800
Speaker 2: working in animal models, people realized, yeah, you can put

142
00:09:52,800 --> 00:09:55,720
Speaker 2: electrodes into the brain, and you can get up close

143
00:09:55,760 --> 00:09:58,079
Speaker 2: next to an individual brain cell a neuron, and when

144
00:09:58,080 --> 00:10:01,199
Speaker 2: that neuron's firing, it's genera a big electric field, a

145
00:10:01,240 --> 00:10:03,520
Speaker 2: tiny electric field, but big relative to the electrode right

146
00:10:03,559 --> 00:10:05,160
Speaker 2: next to it, And so.

147
00:10:05,080 --> 00:10:06,520
Speaker 3: We know that that neuron is firing.

148
00:10:06,559 --> 00:10:09,679
Speaker 2: And then there was a whole decades of systems neuroscience

149
00:10:09,679 --> 00:10:13,240
Speaker 2: which was relating those patterns of activity to what typically

150
00:10:13,280 --> 00:10:16,560
Speaker 2: the animal was doing. So a classic example from the

151
00:10:16,559 --> 00:10:20,240
Speaker 2: eighties would be a monkey is moving his arm up

152
00:10:20,320 --> 00:10:22,920
Speaker 2: or down, or left or right, and you can see

153
00:10:22,920 --> 00:10:26,240
Speaker 2: that maybe a neuron fires more when the arm is

154
00:10:26,280 --> 00:10:28,360
Speaker 2: moving to the left, and say, okay, that neuron has

155
00:10:28,360 --> 00:10:30,960
Speaker 2: a left or preferred direction. We're starting to build some

156
00:10:31,400 --> 00:10:34,800
Speaker 2: mental map of how that brain activity relates to movements.

157
00:10:34,800 --> 00:10:37,240
Speaker 2: Of course, it's much more complicated, and the whole field

158
00:10:37,240 --> 00:10:40,679
Speaker 2: of neuroscience is trying to understand how individual neurons and

159
00:10:40,760 --> 00:10:44,920
Speaker 2: hundreds of neurons and whole large assemblies of neurons generate behavior.

160
00:10:45,320 --> 00:10:50,160
Speaker 2: Starting around the two thousands, the field had felt that

161
00:10:50,240 --> 00:10:53,280
Speaker 2: we had enough of a rudimentary understanding of how movement

162
00:10:53,520 --> 00:10:57,200
Speaker 2: is encoded in the brain that this could be used

163
00:10:57,360 --> 00:10:58,719
Speaker 2: for a medical application.

164
00:10:59,520 --> 00:11:01,240
Speaker 3: And kind of in my world.

165
00:11:01,040 --> 00:11:04,440
Speaker 2: That's been focused on restoring movement to people with paralysis.

166
00:11:04,480 --> 00:11:05,400
Speaker 3: So in two.

167
00:11:05,280 --> 00:11:07,600
Speaker 2: Thousand and four it was a big landmark event that

168
00:11:07,760 --> 00:11:10,319
Speaker 2: was when the original brain Gate trial. So this was

169
00:11:10,400 --> 00:11:13,720
Speaker 2: led by John Donahue in Lee Hagberg at Brown University

170
00:11:13,720 --> 00:11:16,240
Speaker 2: in Masteronal Hospital. They put what was called a multi

171
00:11:16,240 --> 00:11:18,880
Speaker 2: electro array, so instead of a single wire like you

172
00:11:19,040 --> 00:11:21,600
Speaker 2: mentioned in the beginning, now imagine a hundred of those

173
00:11:21,600 --> 00:11:24,959
Speaker 2: little wires kind of all stacked together, recording from thus

174
00:11:25,040 --> 00:11:29,240
Speaker 2: about one hundred neurons. And they showed that these arrays

175
00:11:29,280 --> 00:11:31,480
Speaker 2: could be put in a person with paralysis, and even

176
00:11:31,520 --> 00:11:34,400
Speaker 2: though that person hadn't moved in a decade. I think

177
00:11:34,600 --> 00:11:36,559
Speaker 2: the first guy was a young man in his twenties

178
00:11:36,600 --> 00:11:39,559
Speaker 2: who had been paralyzed from the neck down due to

179
00:11:39,600 --> 00:11:42,560
Speaker 2: a knife wound from like a bar fight. So he

180
00:11:42,600 --> 00:11:46,000
Speaker 2: hadn't moved in many, many years. But they put that

181
00:11:46,040 --> 00:11:48,600
Speaker 2: electro array in the motor cortex, the part of the

182
00:11:48,600 --> 00:11:52,199
Speaker 2: brain that normally sends commands to the arm, and when

183
00:11:52,240 --> 00:11:54,680
Speaker 2: he tried to move his arm, lo and behold, those

184
00:11:54,720 --> 00:11:57,960
Speaker 2: neurons fired away. And so kind of the main risk

185
00:11:58,080 --> 00:12:02,080
Speaker 2: had been solved, which is would the brain even still

186
00:12:02,120 --> 00:12:05,040
Speaker 2: try to generate movements because you might think, well, use

187
00:12:05,080 --> 00:12:07,800
Speaker 2: it or lose it. Right, the person's paralyzed, why would

188
00:12:07,800 --> 00:12:10,880
Speaker 2: their brain still generate movement commands. Fortunately it still does,

189
00:12:11,679 --> 00:12:14,640
Speaker 2: and people were able to decode those signals.

190
00:12:14,320 --> 00:12:16,680
Speaker 1: And just as a quick reminder to everybody, the brain

191
00:12:16,800 --> 00:12:18,920
Speaker 1: is saying, okay, I want you to make these movements,

192
00:12:19,000 --> 00:12:21,880
Speaker 1: and then those shoot down down the spinal cord and

193
00:12:21,880 --> 00:12:24,440
Speaker 1: out to the peripheral nervous system and move the muscles.

194
00:12:24,840 --> 00:12:28,200
Speaker 1: And so in this case you're hearing the original command,

195
00:12:28,720 --> 00:12:33,120
Speaker 1: but there's some break in the roadway plunging down the

196
00:12:33,160 --> 00:12:36,120
Speaker 1: spinal cord and out such that the body never gets

197
00:12:36,160 --> 00:12:37,720
Speaker 1: the signals correctly exactly.

198
00:12:37,760 --> 00:12:39,880
Speaker 2: We're bypassing the injury. We're going to the source. So

199
00:12:39,920 --> 00:12:41,320
Speaker 2: where's the command coming from?

200
00:12:41,360 --> 00:12:43,320
Speaker 1: So this was back in two thousand and four, what

201
00:12:43,320 --> 00:12:46,360
Speaker 1: was his name, Matt Nagel. Is that researchers are able

202
00:12:46,400 --> 00:12:49,400
Speaker 1: to listen to what the neurons are intending, and then

203
00:12:49,760 --> 00:12:51,760
Speaker 1: the field has really taken off since then in the

204
00:12:51,800 --> 00:12:56,120
Speaker 1: past two decades. For example, with motor movement, originally it

205
00:12:56,200 --> 00:12:58,680
Speaker 1: was just on a computer screen you could move a

206
00:12:58,679 --> 00:13:03,079
Speaker 1: cursor around. Nowadays people are thinking about Hey, could you

207
00:13:03,160 --> 00:13:06,719
Speaker 1: actually use an exoskeleton to move the arm physically?

208
00:13:07,120 --> 00:13:09,840
Speaker 3: Yeah, or even stimulate those paralyzed muscles.

209
00:13:09,880 --> 00:13:14,880
Speaker 2: So there's these functional electrical stimulation systems or epidural spinal stimulation,

210
00:13:15,000 --> 00:13:17,959
Speaker 2: both for walking and for the arm. So you can

211
00:13:18,320 --> 00:13:20,800
Speaker 2: really close the loop. You can decode what movement the

212
00:13:20,840 --> 00:13:21,559
Speaker 2: person's trying to make.

213
00:13:21,520 --> 00:13:21,559
Speaker 3: It.

214
00:13:21,600 --> 00:13:23,960
Speaker 2: Oh, they're trying to move their arm forward to grab something,

215
00:13:24,559 --> 00:13:26,960
Speaker 2: and then you can have that move a robotic arm.

216
00:13:27,240 --> 00:13:29,880
Speaker 2: You could have that move an exoskeleton, or if they

217
00:13:29,920 --> 00:13:33,480
Speaker 2: also have a stimulator that's implanted under the skin with

218
00:13:33,559 --> 00:13:36,480
Speaker 2: wires going to the muscles or going outside of the spine,

219
00:13:36,679 --> 00:13:39,880
Speaker 2: you can stimulate the body and actually have the person's

220
00:13:39,880 --> 00:13:44,200
Speaker 2: own formally paralyzed muscles make that movement. It's not at

221
00:13:44,240 --> 00:13:46,280
Speaker 2: the level that you or I let a healthy person

222
00:13:46,320 --> 00:13:48,560
Speaker 2: is moving their arm, but it does work. There's been

223
00:13:48,559 --> 00:13:51,280
Speaker 2: some really amazing studies in the last decade doing that.

224
00:13:51,480 --> 00:13:54,080
Speaker 1: Yeah, exactly right, Okay, great, So that's how people have

225
00:13:54,160 --> 00:13:58,679
Speaker 1: been using brain computer interfaces to move a paralyzed body. Now,

226
00:13:58,760 --> 00:14:01,800
Speaker 1: something that several groups have gotten interested in in recent

227
00:14:01,880 --> 00:14:05,480
Speaker 1: years is what if somebody can't speak anymore? So, what

228
00:14:05,520 --> 00:14:08,040
Speaker 1: are the reasons. First of all, that somebody can't speak.

229
00:14:08,360 --> 00:14:11,960
Speaker 2: So one common one is neurodegenerative diseases like ALS. So

230
00:14:12,040 --> 00:14:16,000
Speaker 2: ALS is a terrible disease, hemiotrophic lateral sclerosis, right and

231
00:14:16,080 --> 00:14:18,839
Speaker 2: right now there's no cure. We can't stop it with

232
00:14:19,240 --> 00:14:21,240
Speaker 2: a drug or other therapy.

233
00:14:21,120 --> 00:14:22,560
Speaker 1: Also known as Luke Gerrig's disease.

234
00:14:22,600 --> 00:14:26,200
Speaker 2: That's right, yeah, and almost everyone who has ALS will

235
00:14:26,240 --> 00:14:28,960
Speaker 2: gradually lose the ability to move their body. But also

236
00:14:29,080 --> 00:14:32,640
Speaker 2: that means what we call the speech articulators, so their lips,

237
00:14:32,680 --> 00:14:35,760
Speaker 2: their jaw, their tongue, their diaphragm, and so their speech

238
00:14:35,800 --> 00:14:39,120
Speaker 2: becomes harder and harder to understand, and eventually you wind

239
00:14:39,200 --> 00:14:41,480
Speaker 2: up what's called locked in, so really not able to

240
00:14:41,520 --> 00:14:44,840
Speaker 2: move at all. And of course this is a terrible situation.

241
00:14:45,680 --> 00:14:48,800
Speaker 2: And if there were a way to restore the ability

242
00:14:48,840 --> 00:14:53,480
Speaker 2: to communicate, so like before decoding not now not they

243
00:14:53,560 --> 00:14:55,480
Speaker 2: are movements that're trying to make, or the leg movements,

244
00:14:55,520 --> 00:14:57,280
Speaker 2: but what are the words that're trying to make, or

245
00:14:57,280 --> 00:14:59,160
Speaker 2: what are the movements of those articulars that they're trying

246
00:14:59,160 --> 00:15:02,600
Speaker 2: to make. What's are they trying to produce? Then we

247
00:15:02,640 --> 00:15:05,680
Speaker 2: can have this person communicate again and talk again through

248
00:15:05,720 --> 00:15:06,160
Speaker 2: a computer.

249
00:15:06,440 --> 00:15:08,520
Speaker 1: If you want to figure out what somebody is trying

250
00:15:08,560 --> 00:15:11,120
Speaker 1: to say, where do you put the electrodes?

251
00:15:11,360 --> 00:15:13,400
Speaker 3: Yeah, and that is the big question. So there are

252
00:15:13,400 --> 00:15:14,200
Speaker 3: a lot of ideas.

253
00:15:14,240 --> 00:15:16,720
Speaker 2: One idea would be the broker's area, which was thought

254
00:15:16,760 --> 00:15:21,200
Speaker 2: to plan speech. Another idea would be the motor cortex,

255
00:15:21,240 --> 00:15:26,440
Speaker 2: which would be kind of the last planning to command generation.

256
00:15:26,520 --> 00:15:28,440
Speaker 2: So the part of the brain that's really sending signals

257
00:15:28,480 --> 00:15:32,640
Speaker 2: to the muscles. And then there's a wide part of

258
00:15:32,720 --> 00:15:34,880
Speaker 2: the brain that are called the language network.

259
00:15:34,920 --> 00:15:36,200
Speaker 3: So this is the temporal lobe.

260
00:15:36,800 --> 00:15:39,760
Speaker 2: It's canonically thought of for perceiving language, but also heavily

261
00:15:39,760 --> 00:15:41,840
Speaker 2: involved in producing language. So there are a lot of

262
00:15:41,920 --> 00:15:46,400
Speaker 2: possible choices. One of the challenges for developing a speech

263
00:15:46,400 --> 00:15:49,840
Speaker 2: ne or prosthesis is there's no animal model. So when

264
00:15:50,240 --> 00:15:52,760
Speaker 2: the field was trying to have people walk again or

265
00:15:52,760 --> 00:15:55,360
Speaker 2: people move their arms again, we had a huge head

266
00:15:55,360 --> 00:15:58,160
Speaker 2: start because you could say, okay, where can you code

267
00:15:58,440 --> 00:16:01,040
Speaker 2: the walking or the arm moved of a rat or

268
00:16:01,080 --> 00:16:04,720
Speaker 2: a monkey or another animal. Well, animals don't talk, they

269
00:16:04,720 --> 00:16:09,360
Speaker 2: don't have language, so we don't have that kind of

270
00:16:09,400 --> 00:16:12,960
Speaker 2: guidance for us, and what we do have are less

271
00:16:13,120 --> 00:16:16,520
Speaker 2: precise measurements from other humans. A lot of the really

272
00:16:16,600 --> 00:16:19,080
Speaker 2: important work from the last decade or twenty years was

273
00:16:19,440 --> 00:16:23,480
Speaker 2: done with electrocorticography. So people with epilepsy often will have

274
00:16:23,840 --> 00:16:26,760
Speaker 2: electrodes put under their skull, typically on top of their

275
00:16:26,800 --> 00:16:30,400
Speaker 2: brain or even in their brain to for the neurologists

276
00:16:30,400 --> 00:16:31,280
Speaker 2: to identify.

277
00:16:30,880 --> 00:16:32,160
Speaker 3: Where the teacher is coming from.

278
00:16:32,440 --> 00:16:34,040
Speaker 2: But these people are then in the hospital for a

279
00:16:34,040 --> 00:16:36,560
Speaker 2: couple of weeks, and this is a gold mine for

280
00:16:36,720 --> 00:16:39,520
Speaker 2: human neuroscience. A lot of what we know about direct

281
00:16:39,520 --> 00:16:42,760
Speaker 2: brain recordings and how they relate to human specific behaviors,

282
00:16:42,800 --> 00:16:46,480
Speaker 2: whether that's speaking or language, or imagination or memory.

283
00:16:46,760 --> 00:16:48,280
Speaker 3: Or mood, all of these things.

284
00:16:48,440 --> 00:16:51,080
Speaker 2: A lot of that comes from this sort of opportunistic

285
00:16:51,160 --> 00:16:53,240
Speaker 2: recording people who are they're in the hospital anyway, they're

286
00:16:53,320 --> 00:16:55,960
Speaker 2: kind of bored, they're waiting for the neurologists to have

287
00:16:56,120 --> 00:16:58,560
Speaker 2: enough data, and so it's very easy to ask them, hey, do.

288
00:16:58,560 --> 00:17:00,680
Speaker 3: You want to read a sentence off a screen.

289
00:17:00,760 --> 00:17:03,960
Speaker 2: So from that we already knew that this sensory motor cortex.

290
00:17:04,080 --> 00:17:08,879
Speaker 2: So the motor and the sensory cortex was a prime area,

291
00:17:08,960 --> 00:17:12,000
Speaker 2: and in our brain Gate clinical trial, that's where we

292
00:17:12,080 --> 00:17:15,359
Speaker 2: ended up putting electrodes, so in the motor part, basically

293
00:17:15,680 --> 00:17:17,879
Speaker 2: the part of the brain that would typically send commands

294
00:17:17,920 --> 00:17:18,679
Speaker 2: to the muscles.

295
00:17:18,920 --> 00:17:23,359
Speaker 1: Great, so it's essentially like the last train station before

296
00:17:23,400 --> 00:17:27,440
Speaker 1: it plunges down towards the muscles. Okay, so you're eavesdropping

297
00:17:27,480 --> 00:17:31,679
Speaker 1: there and you're sticking these little electrode or raise these

298
00:17:31,680 --> 00:17:34,280
Speaker 1: little square jobs where they have sixty four electrodes on

299
00:17:34,280 --> 00:17:35,960
Speaker 1: the one and four of those.

300
00:17:35,920 --> 00:17:38,560
Speaker 2: We used four of them, so yeah, four all along

301
00:17:38,600 --> 00:17:40,680
Speaker 2: this precentral gyrus.

302
00:17:40,760 --> 00:17:44,640
Speaker 1: So you're listening to these neurons and you're trying to

303
00:17:44,840 --> 00:17:49,760
Speaker 1: decode what the person is intending to say from that.

304
00:17:50,280 --> 00:17:53,600
Speaker 1: And one question, were you worried at the beginning that

305
00:17:53,600 --> 00:17:56,720
Speaker 1: that wouldn't be enough data or did you feel like, look,

306
00:17:56,760 --> 00:17:59,640
Speaker 1: with two hundred fifty six neurons, we can figure out

307
00:17:59,680 --> 00:18:02,240
Speaker 1: what's going on in terms of what was trying to

308
00:18:02,320 --> 00:18:03,080
Speaker 1: be articulated.

309
00:18:03,480 --> 00:18:06,359
Speaker 2: When I started the project, I was pretty worried. So

310
00:18:07,200 --> 00:18:09,360
Speaker 2: kind of the prior work is we had shown that

311
00:18:09,400 --> 00:18:11,679
Speaker 2: with about one hundred electrodes in a different part of

312
00:18:11,720 --> 00:18:14,800
Speaker 2: the brain, the hand part of motor cortex, we could

313
00:18:14,800 --> 00:18:18,479
Speaker 2: decode speech, but very poorly. There I was classifying between

314
00:18:18,480 --> 00:18:22,040
Speaker 2: the thirty nine phonemes in American English, if I recall

315
00:18:22,119 --> 00:18:25,760
Speaker 2: about thirty three percent accuracy, So that's way better than chance.

316
00:18:25,800 --> 00:18:27,960
Speaker 2: It showed there's information, but that is not good enough

317
00:18:27,960 --> 00:18:29,280
Speaker 2: to understand.

318
00:18:28,880 --> 00:18:29,440
Speaker 3: What someone's saying.

319
00:18:29,480 --> 00:18:30,679
Speaker 1: Tell us what a phoneme is.

320
00:18:31,240 --> 00:18:33,720
Speaker 3: A phoneme is a building block of speech.

321
00:18:33,800 --> 00:18:36,240
Speaker 2: So I think most people are familiar with the syllables,

322
00:18:36,560 --> 00:18:38,320
Speaker 2: think of a phoneme as a little bit smaller than that.

323
00:18:38,440 --> 00:18:43,200
Speaker 2: So good, ooh E. Right, there's consonants, there's vowels. Different

324
00:18:43,280 --> 00:18:47,159
Speaker 2: languages have different phonemes, but in English, depending on the

325
00:18:47,160 --> 00:18:50,880
Speaker 2: dialect or accent, between thirty nine forty one. These are

326
00:18:50,960 --> 00:18:53,959
Speaker 2: the typical ways we break down English.

327
00:18:54,000 --> 00:18:57,760
Speaker 1: Got So you're recording from these neurons, and you were saying,

328
00:18:57,760 --> 00:19:00,720
Speaker 1: can I figure out what phoneme person is trying to

329
00:19:00,760 --> 00:19:02,919
Speaker 1: say right now and right now just from looking at

330
00:19:02,960 --> 00:19:04,520
Speaker 1: this array of neural activity?

331
00:19:04,720 --> 00:19:05,600
Speaker 3: That's exactly right.

332
00:19:05,680 --> 00:19:09,040
Speaker 2: And a little bit before that, my colleagues at Stanford,

333
00:19:09,080 --> 00:19:10,720
Speaker 2: and that was also the lab that I did my

334
00:19:10,760 --> 00:19:13,800
Speaker 2: post doctoral training, and so I started that project then

335
00:19:13,840 --> 00:19:17,600
Speaker 2: moved on. They had implanted one hundred and twenty eight

336
00:19:17,720 --> 00:19:22,320
Speaker 2: electrodes in the motor cortex of a woman with als,

337
00:19:22,840 --> 00:19:26,000
Speaker 2: and with that they were able to decode what words

338
00:19:26,000 --> 00:19:29,639
Speaker 2: she was saying with about seventy five percent accuracy with

339
00:19:29,680 --> 00:19:31,920
Speaker 2: a large vocabulary of one hundred and twenty five thousand words.

340
00:19:32,080 --> 00:19:35,520
Speaker 2: So that was a really really exciting moment for the

341
00:19:35,520 --> 00:19:38,000
Speaker 2: field because that was really banging at the door of

342
00:19:38,040 --> 00:19:42,639
Speaker 2: making this useful for general communication. Now, three out of

343
00:19:42,640 --> 00:19:45,719
Speaker 2: four words correct is amazing. It was way better than

344
00:19:45,720 --> 00:19:48,320
Speaker 2: anything that ever been done before. But you can't have

345
00:19:48,359 --> 00:19:50,919
Speaker 2: a conversation that way. It's just too frustrating. There's too

346
00:19:50,920 --> 00:19:51,640
Speaker 2: many mistakes.

347
00:19:52,520 --> 00:19:54,399
Speaker 1: And so when we will give us a sense of

348
00:19:54,400 --> 00:19:57,199
Speaker 1: the type of mistake, So the person is intending to

349
00:19:57,240 --> 00:20:01,119
Speaker 1: say the word brain, but the neural activity is decoded

350
00:20:01,160 --> 00:20:03,440
Speaker 1: by the computer, and the computer says, oh, he's trying

351
00:20:03,440 --> 00:20:05,159
Speaker 1: to say panda bear or whatever.

352
00:20:05,359 --> 00:20:07,800
Speaker 3: Well it could be panda bear, it's more likely.

353
00:20:07,880 --> 00:20:10,480
Speaker 1: So the the.

354
00:20:11,320 --> 00:20:14,600
Speaker 2: Way that these systems work is well, one way they work.

355
00:20:14,680 --> 00:20:17,280
Speaker 2: The way our systems work is we're decoding from neural

356
00:20:17,280 --> 00:20:20,600
Speaker 2: activity to phonemes and then those phonemes get assembled into

357
00:20:20,640 --> 00:20:22,840
Speaker 2: words using a dictionary.

358
00:20:22,440 --> 00:20:23,439
Speaker 3: And a language model.

359
00:20:23,760 --> 00:20:25,720
Speaker 2: And in fact, if you look at a dictionary, there's

360
00:20:25,760 --> 00:20:28,160
Speaker 2: that phonetic spelling which most people don't use but if

361
00:20:28,160 --> 00:20:30,520
Speaker 2: you want to figure out how to actually pronounce a word.

362
00:20:30,520 --> 00:20:31,199
Speaker 3: You can look at that.

363
00:20:31,280 --> 00:20:34,120
Speaker 2: So the types of mistakes it would more likely make

364
00:20:34,240 --> 00:20:36,600
Speaker 2: would be similar sounding words.

365
00:20:36,600 --> 00:20:39,800
Speaker 3: So if someone's trying to say brain, maybe they'd get barn.

366
00:20:40,480 --> 00:20:40,920
Speaker 1: Yeah.

367
00:20:40,960 --> 00:20:44,280
Speaker 2: And in some contexts you can understand, oh, I hurt

368
00:20:44,320 --> 00:20:46,720
Speaker 2: my barn, I think you maybe you know you got

369
00:20:46,760 --> 00:20:49,240
Speaker 2: an accident, you hurt your brain. But if there's enough

370
00:20:49,280 --> 00:20:51,560
Speaker 2: of those, it just kind of breaks down. And the

371
00:20:51,560 --> 00:20:54,320
Speaker 2: analogy I'd give is when you're typing on your smartphone.

372
00:20:54,320 --> 00:20:56,560
Speaker 2: Most of us are a little bit clumsy. We make

373
00:20:56,560 --> 00:20:59,760
Speaker 2: a lot of typos. The autocorrect can help up to

374
00:20:59,800 --> 00:21:02,879
Speaker 2: a point, but there's this sort of steep cliff where

375
00:21:03,160 --> 00:21:06,200
Speaker 2: if we're making too many typos, the autocrack so the

376
00:21:06,280 --> 00:21:08,440
Speaker 2: language model cannot keep up, and all of a sudden

377
00:21:08,720 --> 00:21:10,200
Speaker 2: you just get gibberish coming out.

378
00:21:10,680 --> 00:21:12,920
Speaker 3: So that's kind of where things were.

379
00:21:13,080 --> 00:21:15,280
Speaker 2: You could it wasn't gibberish, right, that's overstating it, but

380
00:21:15,680 --> 00:21:33,400
Speaker 2: it was not there for communication day to day.

381
00:21:33,520 --> 00:21:36,719
Speaker 1: So you worked with a man who is forty five

382
00:21:36,800 --> 00:21:40,000
Speaker 1: years old, if I'm rememory correctly, and he had als

383
00:21:40,240 --> 00:21:43,760
Speaker 1: and hadn't articulated in about five years. Is that right?

384
00:21:43,960 --> 00:21:47,480
Speaker 2: Yet he was severely disarthuric, meaning most people couldn't understand him,

385
00:21:47,840 --> 00:21:51,080
Speaker 2: and he volunteered for this brain gate to clinical trial

386
00:21:51,200 --> 00:21:55,200
Speaker 2: that we are one of four sights of which meant

387
00:21:55,359 --> 00:21:59,600
Speaker 2: that after a bunch of tests and imaging scans and

388
00:21:59,640 --> 00:22:02,600
Speaker 2: other things, once we determined that it was a good

389
00:22:02,640 --> 00:22:04,800
Speaker 2: fit and it was safe to move forward. He'd had

390
00:22:04,800 --> 00:22:08,560
Speaker 2: this surgery where doctor Brandman, my collaudrator, put these four

391
00:22:08,960 --> 00:22:11,600
Speaker 2: multi electro to rays into his speech motor cortex.

392
00:22:12,400 --> 00:22:14,240
Speaker 3: We waited a couple of weeks.

393
00:22:13,920 --> 00:22:16,720
Speaker 2: For everything to heal up, and then we went to

394
00:22:16,760 --> 00:22:19,280
Speaker 2: his house where all of our equipment was already pre staged.

395
00:22:19,840 --> 00:22:23,320
Speaker 2: We literally plugged him in. So there's this system is wired,

396
00:22:23,400 --> 00:22:26,480
Speaker 2: so it's not wireless yet. And the way we started

397
00:22:26,520 --> 00:22:29,320
Speaker 2: it was we needed what's called training data in the

398
00:22:29,359 --> 00:22:32,640
Speaker 2: machine learning sense, so we needed the algorithms to see

399
00:22:33,040 --> 00:22:35,479
Speaker 2: a bunch of examples of him trying to say words,

400
00:22:35,480 --> 00:22:37,600
Speaker 2: and then what the neural activity looked like, and what

401
00:22:37,680 --> 00:22:40,240
Speaker 2: this actually looked like in the room was picture a

402
00:22:40,240 --> 00:22:43,399
Speaker 2: person in a wheelchair looking at a computer screen. We

403
00:22:43,520 --> 00:22:46,480
Speaker 2: put up what seemed like random sentences. The text would appear,

404
00:22:46,480 --> 00:22:48,879
Speaker 2: it would turn green, he would try to speak, and

405
00:22:48,920 --> 00:22:50,639
Speaker 2: then he would stop. And we just did this for

406
00:22:50,640 --> 00:22:53,199
Speaker 2: about thirty minutes. And one of the big questions at

407
00:22:53,240 --> 00:22:55,040
Speaker 2: the time was how much data do you need to

408
00:22:55,040 --> 00:22:58,560
Speaker 2: make this work? And the conventional wisdom would it was

409
00:22:58,560 --> 00:23:01,000
Speaker 2: that it would take a lot of data. Previous studies

410
00:23:01,600 --> 00:23:04,919
Speaker 2: had waited many, many weeks before they tried to decode

411
00:23:04,920 --> 00:23:08,560
Speaker 2: what's someone was trying to say. The AI fields that

412
00:23:08,600 --> 00:23:12,240
Speaker 2: we were borrowing tools from, for example, automated dictation when

413
00:23:12,240 --> 00:23:14,760
Speaker 2: you talk to your smartphone, those models are trained with

414
00:23:15,160 --> 00:23:20,280
Speaker 2: millions of hours so huge scrapes data sets to get

415
00:23:20,280 --> 00:23:24,600
Speaker 2: them to be able to understand speech. But it turned

416
00:23:24,640 --> 00:23:26,720
Speaker 2: out that because we had these electrodes in the part

417
00:23:26,760 --> 00:23:29,600
Speaker 2: of part of the brain that's controlling speech movements, it

418
00:23:29,640 --> 00:23:31,720
Speaker 2: has what's called a very high signal to noise ratio.

419
00:23:31,800 --> 00:23:35,800
Speaker 2: There's a really clear signal about what movements the body's

420
00:23:35,840 --> 00:23:38,600
Speaker 2: trying to make and thus what sounds is trying to produce.

421
00:23:39,040 --> 00:23:42,080
Speaker 2: And so after just thirty minutes of him reading these sentences,

422
00:23:42,480 --> 00:23:44,680
Speaker 2: we were looking at our little dashboard on the side

423
00:23:44,680 --> 00:23:46,800
Speaker 2: on our computers and it was showing us what we

424
00:23:46,880 --> 00:23:48,879
Speaker 2: call the word error rate. Or the phoneme error rate,

425
00:23:49,000 --> 00:23:51,920
Speaker 2: so how many words or phonemes were being incorrectly decoded.

426
00:23:52,359 --> 00:23:54,360
Speaker 2: And we saw that that was at the point where

427
00:23:54,359 --> 00:23:56,159
Speaker 2: we thought, okay, this thing can actually work, and so

428
00:23:56,200 --> 00:23:58,399
Speaker 2: we said, okay, now we're gonna do something very special.

429
00:23:58,480 --> 00:24:01,399
Speaker 2: We're gonna kind of flipless, which so to speak, and

430
00:24:01,480 --> 00:24:03,480
Speaker 2: now as you try to speak, you're going to see

431
00:24:03,480 --> 00:24:05,800
Speaker 2: words hopefully appearing at the bottom of the screen. And

432
00:24:05,840 --> 00:24:08,960
Speaker 2: we have a cool video of this, and so everyone's

433
00:24:09,000 --> 00:24:12,920
Speaker 2: kind of holding their breath and very excited, and the

434
00:24:12,960 --> 00:24:15,439
Speaker 2: prompt appeared, and he tries to speak, and the first

435
00:24:15,440 --> 00:24:19,560
Speaker 2: two words appeared correctly, and actually, at that point everyone

436
00:24:19,800 --> 00:24:22,480
Speaker 2: broke out in tears and laughter and clapping.

437
00:24:22,520 --> 00:24:23,720
Speaker 3: We actually paused.

438
00:24:23,359 --> 00:24:26,720
Speaker 2: For a few minutes and hugs, and his family was

439
00:24:26,720 --> 00:24:29,160
Speaker 2: there to watch it, in a really amazing moment, and

440
00:24:29,200 --> 00:24:31,520
Speaker 2: then we said, all right, let's get back to work,

441
00:24:31,880 --> 00:24:34,520
Speaker 2: and we kept going. And on that day we had

442
00:24:34,520 --> 00:24:36,840
Speaker 2: set a relatively modest goal. So we were using what's

443
00:24:36,840 --> 00:24:40,120
Speaker 2: called a fifty word vocabulary, meaning the sentences he could

444
00:24:40,119 --> 00:24:43,199
Speaker 2: say with it were restricted to fifty words, and you

445
00:24:43,200 --> 00:24:46,439
Speaker 2: can still say a few things, and that's obviously not

446
00:24:46,760 --> 00:24:49,480
Speaker 2: pragmatically useful, but that was to just to get going.

447
00:24:50,000 --> 00:24:52,960
Speaker 2: We had less than a one percent error rate using

448
00:24:53,040 --> 00:24:55,720
Speaker 2: this fifty word vocabulary, so almost every word was correct.

449
00:24:56,359 --> 00:24:56,960
Speaker 3: That was huge.

450
00:24:56,960 --> 00:25:01,280
Speaker 2: So we'd already established that, like some previous clinical throw participants,

451
00:25:01,640 --> 00:25:03,800
Speaker 2: his brain was still active when he was trying to speak.

452
00:25:03,880 --> 00:25:05,879
Speaker 2: So good, all right, that was the big one of

453
00:25:05,920 --> 00:25:09,240
Speaker 2: the bigger risks. Were we getting good in neural signals

454
00:25:09,240 --> 00:25:12,320
Speaker 2: from these electroder arrays? Yes, we were getting beautiful neural signals,

455
00:25:12,359 --> 00:25:14,399
Speaker 2: in fact, some of the best I've seen in my career.

456
00:25:14,640 --> 00:25:16,840
Speaker 2: And then did we need a ton of data? And

457
00:25:17,119 --> 00:25:19,320
Speaker 2: the answer was no, we were getting enough that we

458
00:25:19,359 --> 00:25:22,840
Speaker 2: could train these machine learning algorithms to map the neural

459
00:25:22,880 --> 00:25:24,919
Speaker 2: activity patterns to the words okay.

460
00:25:24,920 --> 00:25:27,320
Speaker 1: And for the listeners, I'm going to link the video

461
00:25:27,600 --> 00:25:30,240
Speaker 1: which shows when the family started to cry and so

462
00:25:30,320 --> 00:25:33,720
Speaker 1: I found that very moving. And so how long will

463
00:25:33,760 --> 00:25:39,400
Speaker 1: these electrodes last? And you'd be able to get good

464
00:25:39,480 --> 00:25:40,439
Speaker 1: signal out of this?

465
00:25:40,600 --> 00:25:44,480
Speaker 2: For Casey that is a key question, and the answers

466
00:25:44,520 --> 00:25:47,600
Speaker 2: we just don't know. So at this point he has

467
00:25:47,640 --> 00:25:50,240
Speaker 2: had this for about two years. We just had a

468
00:25:50,240 --> 00:25:53,760
Speaker 2: preprint a few months ago showing that out past six

469
00:25:53,840 --> 00:25:56,760
Speaker 2: hundred and fifty days the system is still going strong.

470
00:25:56,880 --> 00:26:00,959
Speaker 2: So this is huge because there was always some concern

471
00:26:01,000 --> 00:26:03,639
Speaker 2: that maybe these electrodes would stop recording neurons after a

472
00:26:03,680 --> 00:26:05,680
Speaker 2: few months or.

473
00:26:06,080 --> 00:26:09,000
Speaker 1: And why it's because of scar tissue building up around

474
00:26:09,040 --> 00:26:09,800
Speaker 1: the electrode.

475
00:26:09,880 --> 00:26:12,520
Speaker 2: There are a lot of potential factors. So yeah, whenever

476
00:26:12,560 --> 00:26:15,720
Speaker 2: you have a foreign body in the brain, the body

477
00:26:15,760 --> 00:26:19,280
Speaker 2: in the brain does not want that thing, So scar

478
00:26:19,359 --> 00:26:22,240
Speaker 2: tissue can form, can be at the microscale, just around

479
00:26:22,280 --> 00:26:25,800
Speaker 2: the electrode tip, which makes it harder to record individual neurons.

480
00:26:25,680 --> 00:26:28,720
Speaker 2: That sort of think of it like you're moving further

481
00:26:28,760 --> 00:26:31,879
Speaker 2: away from someone you're listening to, or there's padding between

482
00:26:31,920 --> 00:26:33,600
Speaker 2: you and them. It kind of it muffles the signal.

483
00:26:34,200 --> 00:26:36,000
Speaker 2: It could be at a more of a macro scale

484
00:26:36,000 --> 00:26:38,680
Speaker 2: where it can actually pull the electrodes out of the brain,

485
00:26:38,720 --> 00:26:40,360
Speaker 2: and that's happened in some other studies.

486
00:26:40,440 --> 00:26:42,440
Speaker 1: The way that your skin pushes a splinter out.

487
00:26:42,600 --> 00:26:45,679
Speaker 2: Yeah, I think that's a good analogy. So that's on

488
00:26:45,760 --> 00:26:49,960
Speaker 2: the biological response. Also, these are electrodes, so the materials

489
00:26:50,000 --> 00:26:53,240
Speaker 2: can fail, The insulation can fail over time, the metal

490
00:26:53,280 --> 00:26:55,760
Speaker 2: can get kind of chipped away or even away at

491
00:26:56,119 --> 00:27:01,000
Speaker 2: the wires, could disconnect, and there's a lot of failure modes,

492
00:27:01,359 --> 00:27:05,120
Speaker 2: but in this case, the records offar is really really encouraging.

493
00:27:05,160 --> 00:27:08,639
Speaker 2: So two years out, it's working great. The accuracy has

494
00:27:08,640 --> 00:27:10,920
Speaker 2: actually gotten better, and our preprint is now ninety nine

495
00:27:10,920 --> 00:27:13,560
Speaker 2: percent accurate, both because we have more data and we've

496
00:27:13,600 --> 00:27:15,760
Speaker 2: had more time to just improve the algorithms and keep

497
00:27:15,840 --> 00:27:18,800
Speaker 2: trying new things. And he is now using this as

498
00:27:18,800 --> 00:27:20,280
Speaker 2: his primary means of communication.

499
00:27:20,560 --> 00:27:22,600
Speaker 1: And so a couple of things. One is, when you

500
00:27:22,680 --> 00:27:25,359
Speaker 1: decode the neural activity, you could just print that as

501
00:27:25,480 --> 00:27:27,879
Speaker 1: words on the screen, but you guys went a step further.

502
00:27:28,520 --> 00:27:32,640
Speaker 2: Yeah, So in our first few months, what we did

503
00:27:32,720 --> 00:27:34,919
Speaker 2: is called text to speech, So the words would appear

504
00:27:34,960 --> 00:27:38,040
Speaker 2: as text on the screen initially, and then when a

505
00:27:38,080 --> 00:27:40,199
Speaker 2: whole utter and so a sentence or it could be

506
00:27:40,200 --> 00:27:43,440
Speaker 2: a whole paragraph, he would use his eyes to look

507
00:27:43,440 --> 00:27:45,440
Speaker 2: at a button on the screen and basically there's a

508
00:27:45,480 --> 00:27:48,320
Speaker 2: done button, and after he hits the done button, the

509
00:27:48,440 --> 00:27:51,600
Speaker 2: computer will read out loud what he said, and we

510
00:27:51,680 --> 00:27:53,720
Speaker 2: basically made a deep fake of his voice, so it

511
00:27:53,800 --> 00:27:56,560
Speaker 2: sounds a lot like he did before he got als.

512
00:27:56,840 --> 00:27:59,440
Speaker 2: It's not perfect, but it really does sound quite a

513
00:27:59,440 --> 00:28:02,280
Speaker 2: lot like him. Technology has progressed a lot, even in

514
00:28:02,280 --> 00:28:04,879
Speaker 2: the last couple of years. Most of the time people

515
00:28:04,880 --> 00:28:08,400
Speaker 2: worry about all the ill uses of faking someone's voice,

516
00:28:08,400 --> 00:28:10,640
Speaker 2: but this is maybe one of the few cases where

517
00:28:10,640 --> 00:28:12,000
Speaker 2: it's actually a really wonderful thing.

518
00:28:12,400 --> 00:28:15,560
Speaker 1: So you got his voice from videos when he was younger,

519
00:28:15,560 --> 00:28:17,159
Speaker 1: before the als had set in.

520
00:28:17,480 --> 00:28:19,479
Speaker 2: Yeah, we asked him and his family and they provided

521
00:28:19,560 --> 00:28:21,200
Speaker 2: us a bunch of things. And actually he had done

522
00:28:21,200 --> 00:28:25,440
Speaker 2: a podcast before, so we had really good material.

523
00:28:25,640 --> 00:28:29,440
Speaker 1: So when he thinks of a sentence, the neural activities decoded,

524
00:28:29,480 --> 00:28:34,440
Speaker 1: the sentence gets reconstructed, and then you turn it into

525
00:28:34,520 --> 00:28:37,200
Speaker 1: his voice. Yes, now that's what you showed in twenty

526
00:28:37,240 --> 00:28:39,600
Speaker 1: twenty four, and you just recently had a paper five

527
00:28:39,600 --> 00:28:41,680
Speaker 1: months ago or so. Tell us about that.

528
00:28:42,120 --> 00:28:45,360
Speaker 2: Yeah, So everything before, even though it could be said

529
00:28:45,400 --> 00:28:48,920
Speaker 2: out loud, ultimately the informations in the form of text.

530
00:28:49,880 --> 00:28:52,320
Speaker 2: And I think we can all appreciate that a lot

531
00:28:52,400 --> 00:28:54,360
Speaker 2: gets lost just through texts.

532
00:28:55,600 --> 00:28:56,959
Speaker 3: There's no intonation.

533
00:28:57,200 --> 00:29:02,239
Speaker 2: You can't indicate that maybe you're being sarcastic. It's less expressive. Right,

534
00:29:02,240 --> 00:29:05,120
Speaker 2: There's a lot of rich nuance that we all convey

535
00:29:05,520 --> 00:29:08,400
Speaker 2: in our voice and through text that's lost, and the

536
00:29:08,440 --> 00:29:11,960
Speaker 2: other problem is the latency or the immediacy. So if

537
00:29:12,040 --> 00:29:14,600
Speaker 2: I was talking to you and I could only write,

538
00:29:15,240 --> 00:29:18,040
Speaker 2: it would be very easy for you to accidentally interrupt me,

539
00:29:18,520 --> 00:29:20,480
Speaker 2: or to just not for me not to be able

540
00:29:20,480 --> 00:29:23,160
Speaker 2: to get a word in, because by the time I've

541
00:29:23,360 --> 00:29:25,800
Speaker 2: finished a sentence and selected a bund to speak it

542
00:29:25,800 --> 00:29:28,360
Speaker 2: out loud, maybe you've already moved on to the next topic.

543
00:29:28,440 --> 00:29:31,880
Speaker 2: Maybe if there's other people in the room, they're talking right. So,

544
00:29:32,240 --> 00:29:34,400
Speaker 2: for all of these reasons, we really wanted to do

545
00:29:34,760 --> 00:29:36,240
Speaker 2: not what we call brain to text, but what we

546
00:29:36,280 --> 00:29:39,200
Speaker 2: call brain to voice, and that means go immediately from

547
00:29:39,240 --> 00:29:42,880
Speaker 2: neuroactivity to sound. This is a hard problem for a

548
00:29:42,880 --> 00:29:45,000
Speaker 2: lot of reasons, one of which is it has to

549
00:29:45,000 --> 00:29:48,160
Speaker 2: be in super fast. You want sound to happen within

550
00:29:48,200 --> 00:29:52,160
Speaker 2: about thirty millisecond. That's kind of matching the natural latency

551
00:29:52,200 --> 00:29:56,120
Speaker 2: of brain to moving the muscles to vibrating air that

552
00:29:56,600 --> 00:30:00,520
Speaker 2: someone can hear. And so because of that, first of all,

553
00:30:00,520 --> 00:30:03,200
Speaker 2: we had to decode these neuro signals very quickly. It

554
00:30:03,320 --> 00:30:06,000
Speaker 2: limits the kind of algorithms we can use. We have

555
00:30:06,120 --> 00:30:08,400
Speaker 2: less data to work with. Right, you can't look into

556
00:30:08,440 --> 00:30:11,520
Speaker 2: the future, there's no autocorrect. You can't look at the

557
00:30:11,640 --> 00:30:15,200
Speaker 2: entire sentence to figure out based on context, like, Oh,

558
00:30:15,200 --> 00:30:17,959
Speaker 2: I reached down to pet the cot. No, you probably

559
00:30:17,960 --> 00:30:20,960
Speaker 2: meant kat because you don't usually pet a cot. You

560
00:30:21,000 --> 00:30:23,720
Speaker 2: can't do that if you're doing brain to voice. As

561
00:30:23,720 --> 00:30:25,640
Speaker 2: soon as you try to say I, you need to

562
00:30:25,640 --> 00:30:29,160
Speaker 2: have the sound eye reached. Right. It just has to

563
00:30:29,360 --> 00:30:33,640
Speaker 2: flow constantly. But we were able to, through a bunch

564
00:30:33,640 --> 00:30:38,200
Speaker 2: of complicated engineering work, get really far in there. And

565
00:30:38,400 --> 00:30:40,240
Speaker 2: where the state of the art in that paper that

566
00:30:40,280 --> 00:30:43,719
Speaker 2: you're referring to is is it is very immediate, So

567
00:30:43,760 --> 00:30:49,200
Speaker 2: the latency is under thirty milliseconds, and it's mostly intelligible,

568
00:30:49,200 --> 00:30:51,920
Speaker 2: but not consistently intelligible. So about fifty six percent of

569
00:30:51,960 --> 00:30:56,120
Speaker 2: words could be understood by someone. It's a big step forward,

570
00:30:56,160 --> 00:30:58,720
Speaker 2: but it's not good enough for daily use. Right. I

571
00:30:58,760 --> 00:31:01,000
Speaker 2: already said earlier that we out of four words is

572
00:31:01,040 --> 00:31:03,440
Speaker 2: not good enough, So you know, one out of two

573
00:31:03,480 --> 00:31:04,840
Speaker 2: words is definitely not good enough.

574
00:31:05,040 --> 00:31:07,440
Speaker 1: So when there's a mistake, what kind of mistake is it?

575
00:31:07,480 --> 00:31:11,920
Speaker 1: Is it barn for brain and therefore sort of intelligible,

576
00:31:12,000 --> 00:31:13,080
Speaker 1: or is it is it worse than that?

577
00:31:13,720 --> 00:31:16,800
Speaker 2: Yeah, it tends to sound like slurry speech, or maybe

578
00:31:16,840 --> 00:31:20,480
Speaker 2: like if someone's mumbling, so sometimes you can get the

579
00:31:20,560 --> 00:31:23,040
Speaker 2: gist of it. The length tends to be the same

580
00:31:23,040 --> 00:31:26,120
Speaker 2: because it's still capturing we call the envelope of speech.

581
00:31:26,200 --> 00:31:28,440
Speaker 2: So if you're saying a short word or a long word,

582
00:31:28,640 --> 00:31:31,800
Speaker 2: that comes through it very clearly, but maybe some of

583
00:31:31,800 --> 00:31:33,640
Speaker 2: the phonemes are a little garbled, and so you can't

584
00:31:33,840 --> 00:31:35,680
Speaker 2: tell exactly what's being said.

585
00:31:35,920 --> 00:31:39,960
Speaker 1: Got it, Because each phoneme that the brain is encoding for,

586
00:31:40,160 --> 00:31:43,040
Speaker 1: you're translating that right away. Thirty milli seconds later that's

587
00:31:43,080 --> 00:31:44,080
Speaker 1: coming out of the speaker.

588
00:31:44,360 --> 00:31:47,080
Speaker 2: Yeah, we just don't have enough signal to noise ratio.

589
00:31:47,080 --> 00:31:49,160
Speaker 2: We don't have enough precisions. So it's like if you

590
00:31:49,200 --> 00:31:52,640
Speaker 2: have a really bad digital camera, really grainy camera, and

591
00:31:52,680 --> 00:31:55,120
Speaker 2: you're trying to parse the scene. You know, sometimes you

592
00:31:55,160 --> 00:31:56,920
Speaker 2: can see what's going on, and other times you just

593
00:31:57,080 --> 00:32:00,040
Speaker 2: can't quite make out. I know that is that a

594
00:32:00,080 --> 00:32:01,640
Speaker 2: person or a ball?

595
00:32:01,760 --> 00:32:01,959
Speaker 3: Is that?

596
00:32:02,040 --> 00:32:05,560
Speaker 2: You know? What does that word say? If it's really grainy,

597
00:32:05,880 --> 00:32:07,720
Speaker 2: you just can't see so well. And although we have

598
00:32:07,760 --> 00:32:10,040
Speaker 2: two hundred and fifty six electros, which sounds like a lot,

599
00:32:10,680 --> 00:32:14,000
Speaker 2: the brain has almost one hundred billion neurons. There's probably

600
00:32:14,120 --> 00:32:17,320
Speaker 2: multiple billions that are involved in just speech and language.

601
00:32:17,360 --> 00:32:20,120
Speaker 2: So in some ways as a miracle that works at all,

602
00:32:20,160 --> 00:32:23,120
Speaker 2: that we're sampling from such a small number of neurons

603
00:32:23,360 --> 00:32:26,040
Speaker 2: and able to reconstruct the sounds that the person's trying

604
00:32:26,040 --> 00:32:26,280
Speaker 2: to make.

605
00:32:27,200 --> 00:32:30,280
Speaker 1: And if I'm remembering in that paper, you also showed

606
00:32:31,440 --> 00:32:32,800
Speaker 1: sort of short singing.

607
00:32:33,120 --> 00:32:37,240
Speaker 2: Yeah, So we wanted to demonstrate that this approach could

608
00:32:37,320 --> 00:32:41,480
Speaker 2: do more than just transmit the words, because we kind

609
00:32:41,480 --> 00:32:44,000
Speaker 2: of already had that with brain to text. Now it

610
00:32:44,040 --> 00:32:46,520
Speaker 2: could do it immediately, so that solves that interruption or

611
00:32:46,560 --> 00:32:49,040
Speaker 2: being heard right away problem. But we wanted to provide

612
00:32:49,040 --> 00:32:51,480
Speaker 2: a proof of concept that this could also be expressive,

613
00:32:51,600 --> 00:32:54,479
Speaker 2: so we had a couple experiments that did that. In

614
00:32:54,520 --> 00:32:56,400
Speaker 2: one of them, he was asked to say sentences as

615
00:32:56,440 --> 00:32:59,440
Speaker 2: either a question or a statement. And in English, when

616
00:32:59,440 --> 00:33:01,520
Speaker 2: we ask a question, can we increase the pitch at

617
00:33:01,560 --> 00:33:03,720
Speaker 2: the end, So he was able to do that. We

618
00:33:03,760 --> 00:33:06,400
Speaker 2: had him emphasize specific words, and you know, you use

619
00:33:06,480 --> 00:33:09,000
Speaker 2: that to change the meaning of what you're saying. So

620
00:33:09,160 --> 00:33:12,360
Speaker 2: this is classic from a different study, sentence that you

621
00:33:12,360 --> 00:33:14,560
Speaker 2: can say in seven different ways, which is I never

622
00:33:14,600 --> 00:33:17,480
Speaker 2: said she stole my money. Now I can say I

623
00:33:17,520 --> 00:33:20,440
Speaker 2: never said she stole my money. I never said she

624
00:33:20,560 --> 00:33:23,880
Speaker 2: stole my money. Right, I'm slightly changing the connotation depending

625
00:33:23,920 --> 00:33:25,920
Speaker 2: on which word I'm stressing. And so we had a

626
00:33:25,960 --> 00:33:28,800
Speaker 2: task where he said that sentence emphasizing all the different

627
00:33:28,800 --> 00:33:30,760
Speaker 2: words and lo and behold.

628
00:33:30,800 --> 00:33:30,960
Speaker 1: Yes.

629
00:33:31,000 --> 00:33:34,200
Speaker 2: From the neuroactivity, we could identify which word he was stressing.

630
00:33:34,240 --> 00:33:36,280
Speaker 2: And so then we had another task where we would

631
00:33:36,320 --> 00:33:38,120
Speaker 2: give him a sentence and we would capitalize a word

632
00:33:38,400 --> 00:33:40,080
Speaker 2: and he was supposed to emphasize that. And then the

633
00:33:40,120 --> 00:33:42,640
Speaker 2: last one is what you were referring to is we

634
00:33:42,720 --> 00:33:47,080
Speaker 2: call a simple singing task. So it was only three notes,

635
00:33:47,200 --> 00:33:49,640
Speaker 2: but basically he could say whatever he wanted to say,

636
00:33:49,640 --> 00:33:52,000
Speaker 2: but at three different pitch levels, so you could say,

637
00:33:52,000 --> 00:33:54,960
Speaker 2: you know, like bah bah bah or like you know,

638
00:33:55,320 --> 00:34:00,280
Speaker 2: la law da. So that task he was able to

639
00:34:00,360 --> 00:34:03,680
Speaker 2: do quite well. He's not going to be singing in

640
00:34:03,720 --> 00:34:06,880
Speaker 2: the opera yet, but it shows the path forward and

641
00:34:07,520 --> 00:34:10,440
Speaker 2: where our lab and many others are working now is

642
00:34:10,800 --> 00:34:12,560
Speaker 2: how do we build on this? So does that mean

643
00:34:12,960 --> 00:34:17,360
Speaker 2: better algorithms? There's always new innovations in the artificial intelligence

644
00:34:17,360 --> 00:34:20,200
Speaker 2: world and just neuroscience making sense of these signals.

645
00:34:20,440 --> 00:34:21,960
Speaker 3: Does that mean putting more electrodes?

646
00:34:22,000 --> 00:34:22,080
Speaker 1: In.

647
00:34:22,200 --> 00:34:24,480
Speaker 2: Certainly that's of interest, and there's a lot of really

648
00:34:24,480 --> 00:34:28,320
Speaker 2: exciting work happening in there. Does that mean maybe putting

649
00:34:28,320 --> 00:34:32,040
Speaker 2: electrodes in additional parts of the brain, so kind of

650
00:34:32,040 --> 00:34:35,160
Speaker 2: at a simplistic level, people think of left versus right

651
00:34:35,200 --> 00:34:37,600
Speaker 2: brain as having some differences with maybe more of these

652
00:34:37,760 --> 00:34:41,680
Speaker 2: what are called parlinguistic elements of voice encoded more on

653
00:34:41,719 --> 00:34:44,239
Speaker 2: the right side of the brain. That's something we'd like

654
00:34:44,320 --> 00:34:46,120
Speaker 2: to find out and we hope to in the future,

655
00:34:46,880 --> 00:34:48,799
Speaker 2: or do we need to put it in other parts

656
00:34:48,840 --> 00:34:50,160
Speaker 2: of the speech network.

657
00:34:50,200 --> 00:34:53,040
Speaker 1: By the way, just to flesh that out for listeners.

658
00:34:53,719 --> 00:34:55,160
Speaker 1: You know, on the left side of the brain, you've

659
00:34:55,200 --> 00:34:58,880
Speaker 1: got a lot involved with language. When people get damage there,

660
00:34:59,239 --> 00:35:03,680
Speaker 1: they let's say, lose the ability to articulate, to produce sentences,

661
00:35:03,680 --> 00:35:07,560
Speaker 1: to understand census. But when people get damage in equivalent

662
00:35:07,600 --> 00:35:10,239
Speaker 1: areas mirror images on the right side, they can get

663
00:35:10,239 --> 00:35:12,840
Speaker 1: what's called a musia, which is the inability to understand

664
00:35:12,960 --> 00:35:16,319
Speaker 1: music anymore. Because as you say, that's where intonation, the

665
00:35:16,400 --> 00:35:20,839
Speaker 1: prosity of language seems to be encoded. So good, this

666
00:35:20,920 --> 00:35:23,040
Speaker 1: is a good segue into the future, then, which is

667
00:35:24,040 --> 00:35:27,600
Speaker 1: first of all, I'm curious what you think is the

668
00:35:27,680 --> 00:35:31,440
Speaker 1: answer you just posed. Is it getting better electrodes, more electrodes,

669
00:35:31,520 --> 00:35:34,319
Speaker 1: is it getting better algorithms? Is there a limitation in

670
00:35:34,360 --> 00:35:39,880
Speaker 1: the signals and noise ratio? Where's the lowest hanging fruit

671
00:35:39,960 --> 00:35:41,239
Speaker 1: for getting improvements? Here?

672
00:35:41,760 --> 00:35:44,279
Speaker 3: Can I go with d all of the above? I

673
00:35:44,320 --> 00:35:46,000
Speaker 3: think we do need all of these things.

674
00:35:46,239 --> 00:35:50,560
Speaker 2: So already we are seeing with our data and this

675
00:35:50,600 --> 00:35:54,439
Speaker 2: current participant that with the same electrodes, we are able

676
00:35:54,480 --> 00:35:57,279
Speaker 2: to squeeze more information out with better algorithms and just

677
00:35:57,480 --> 00:35:59,600
Speaker 2: better understanding what the brain is doing. And there's a

678
00:35:59,640 --> 00:36:02,399
Speaker 2: lot going on there. It's not just the movements. We're

679
00:36:02,400 --> 00:36:07,480
Speaker 2: seeing things like neural error signals. We're seeing prosody and

680
00:36:07,520 --> 00:36:10,160
Speaker 2: intonation encoded. Right. All of these things are kind of

681
00:36:10,520 --> 00:36:14,560
Speaker 2: mixed together in these brain signals we're measuring, and there's

682
00:36:14,560 --> 00:36:17,239
Speaker 2: a lot of science that goes into disentangling them and

683
00:36:17,239 --> 00:36:19,000
Speaker 2: figure out what they mean. What are you trying to

684
00:36:19,000 --> 00:36:22,640
Speaker 2: pay attention to for given application. So that's all moving forward,

685
00:36:23,320 --> 00:36:25,200
Speaker 2: and so we're just learning a ton about how the

686
00:36:25,239 --> 00:36:28,920
Speaker 2: human brain produces speech because we didn't have this opportunity

687
00:36:28,960 --> 00:36:31,880
Speaker 2: at this precision before. There's now only a handful of

688
00:36:31,960 --> 00:36:34,719
Speaker 2: humans in the whole world that have had electrodes that

689
00:36:34,760 --> 00:36:37,359
Speaker 2: measure individual neurons as they try to speak. So we're

690
00:36:37,400 --> 00:36:41,160
Speaker 2: learning a lot, but certainly more electrodes is better, So

691
00:36:41,360 --> 00:36:43,400
Speaker 2: in our trial as we move forward, we intend to

692
00:36:43,400 --> 00:36:45,880
Speaker 2: put more electrodes in. There are now multiple companies that

693
00:36:45,920 --> 00:36:49,719
Speaker 2: are building fully implanted intracortical electrodes, so similar type of

694
00:36:49,719 --> 00:36:53,200
Speaker 2: electrodes that go right up to the neurons, but they

695
00:36:53,200 --> 00:36:56,600
Speaker 2: all have a thousand or more electrodes or recording sites.

696
00:36:57,080 --> 00:36:59,000
Speaker 2: So we're talking about at least a four x if

697
00:36:59,040 --> 00:37:03,120
Speaker 2: not more improved in the density or the count of electrodes.

698
00:37:03,120 --> 00:37:05,400
Speaker 2: And I think that's going to make everything work just

699
00:37:05,600 --> 00:37:06,400
Speaker 2: so much better.

700
00:37:06,800 --> 00:37:09,480
Speaker 1: And of course companies were working on making this wireless

701
00:37:09,520 --> 00:37:12,960
Speaker 1: as well, Neurallink being I guess the first one to

702
00:37:13,040 --> 00:37:15,800
Speaker 1: do it, but other companies moving that way as well,

703
00:37:16,360 --> 00:37:19,480
Speaker 1: so that you could have something that's fully packaged and

704
00:37:19,520 --> 00:37:23,040
Speaker 1: a person can just speak with no wires hanging out.

705
00:37:23,360 --> 00:37:25,400
Speaker 3: Yeah, that is very important.

706
00:37:25,400 --> 00:37:29,200
Speaker 2: So the wired systems we have now, they are what

707
00:37:29,320 --> 00:37:32,800
Speaker 2: is available. They're good for research there in some ways simpler.

708
00:37:33,360 --> 00:37:37,000
Speaker 2: They've been shown to be safe for quite a long time,

709
00:37:37,400 --> 00:37:39,799
Speaker 2: but they're limiting right fully implanted is the way to go,

710
00:37:39,840 --> 00:37:42,879
Speaker 2: and we can look at other medical devices. So there's

711
00:37:42,880 --> 00:37:47,240
Speaker 2: these wild photos of pacemakers in the fifties and it

712
00:37:47,320 --> 00:37:50,480
Speaker 2: was basically like a car battery on a cart with

713
00:37:50,640 --> 00:37:53,880
Speaker 2: you some amplifiers and kind of primitive. They're not computers,

714
00:37:53,920 --> 00:37:56,760
Speaker 2: they're electronics, and then there's a wire going to someone's chest.

715
00:37:57,520 --> 00:37:59,880
Speaker 3: It kept them alive and it showed that this worked.

716
00:38:00,400 --> 00:38:03,080
Speaker 2: But of course today millions and millions of people are

717
00:38:03,080 --> 00:38:07,160
Speaker 2: walking around very healthy with pacemakers that are small and

718
00:38:07,200 --> 00:38:10,680
Speaker 2: their packaged and titanium or other very inert safe materials.

719
00:38:11,640 --> 00:38:12,440
Speaker 3: They have battery.

720
00:38:12,600 --> 00:38:15,319
Speaker 2: Some of them now can be wirelessly recharged. So I

721
00:38:15,320 --> 00:38:18,640
Speaker 2: think this is a well trodden path and we're going

722
00:38:18,680 --> 00:38:21,200
Speaker 2: to absolutely see this with brain computer interfaces. They're going

723
00:38:21,239 --> 00:38:23,680
Speaker 2: to be fully implanted, they're going to be wireless. Data

724
00:38:23,719 --> 00:38:26,160
Speaker 2: is going to come out through radio or lasers or

725
00:38:26,160 --> 00:38:28,920
Speaker 2: other means to get data out of the brain, and

726
00:38:29,160 --> 00:38:31,279
Speaker 2: power is going to go in and it's going to

727
00:38:31,280 --> 00:38:31,960
Speaker 2: be great. Great.

728
00:38:32,280 --> 00:38:34,120
Speaker 1: Now, Okay, let me ask you this. A lot of

729
00:38:34,160 --> 00:38:36,799
Speaker 1: people are very familiar with neuralink. They've heard about it.

730
00:38:36,880 --> 00:38:38,839
Speaker 1: Even though as I mentioned, this idea of recording from

731
00:38:38,840 --> 00:38:40,640
Speaker 1: brains has been happening for a very long time.

732
00:38:40,960 --> 00:38:41,120
Speaker 2: Now.

733
00:38:41,120 --> 00:38:45,839
Speaker 1: What neuralink is doing is implanting very tiny electrodes robotically,

734
00:38:46,040 --> 00:38:49,040
Speaker 1: and it's fully implantable, and so that's part of why

735
00:38:49,040 --> 00:38:50,880
Speaker 1: it's famous. But also part of why it's famous this

736
00:38:50,920 --> 00:38:55,040
Speaker 1: is because it's Elon and there's this mystique about it,

737
00:38:55,080 --> 00:38:59,640
Speaker 1: the sort of idea that everyone will someday get a neuralink.

738
00:39:00,280 --> 00:39:03,080
Speaker 1: Now I have my doubts because it's an open head

739
00:39:03,080 --> 00:39:06,280
Speaker 1: surgery still, even though it's with the robot. But let's

740
00:39:06,280 --> 00:39:11,359
Speaker 1: look towards the future in terms of what use would

741
00:39:11,400 --> 00:39:14,720
Speaker 1: it be to have a brain computer interface for somebody

742
00:39:14,760 --> 00:39:16,920
Speaker 1: without a problem speaking or moving.

743
00:39:17,320 --> 00:39:21,080
Speaker 2: Yeah, I don't think that application, the killer app so

744
00:39:21,200 --> 00:39:22,960
Speaker 2: to speak, has been discovered yet.

745
00:39:23,040 --> 00:39:25,719
Speaker 3: You know, there's times where I'm lying.

746
00:39:25,480 --> 00:39:27,080
Speaker 2: In bed and I kind of wish i could send

747
00:39:27,120 --> 00:39:29,000
Speaker 2: a text message without having to reach for my phone.

748
00:39:29,040 --> 00:39:30,759
Speaker 2: But I'm not going to get a brain surgery to

749
00:39:30,800 --> 00:39:32,640
Speaker 2: do that. I'm going to just reach for my phone.

750
00:39:32,920 --> 00:39:36,160
Speaker 2: So what I think we're going to see is a

751
00:39:36,200 --> 00:39:39,680
Speaker 2: widening of the medical applications. So I think there's gonna

752
00:39:39,680 --> 00:39:43,320
Speaker 2: be many, many more medical needs that can be addressed

753
00:39:43,320 --> 00:39:48,440
Speaker 2: with brain technology, whether stroke, things like sustaining memory in

754
00:39:48,480 --> 00:39:52,120
Speaker 2: the longer term, or dealing with age related decline or

755
00:39:52,120 --> 00:39:54,520
Speaker 2: even Alzheimer's. So there's going to be different types of

756
00:39:54,600 --> 00:39:59,000
Speaker 2: BCIs for different problems. But in terms of fully implanted,

757
00:39:59,080 --> 00:40:03,520
Speaker 2: kind of invasivec eyes for really healthy people, no one

758
00:40:03,560 --> 00:40:09,280
Speaker 2: has yet shown a benefit that I think is worthwhile. Now,

759
00:40:09,400 --> 00:40:12,920
Speaker 2: could I imagine it? Certainly one could imagine it. So,

760
00:40:13,600 --> 00:40:15,520
Speaker 2: you know, if you could have a device in your brain,

761
00:40:15,680 --> 00:40:19,160
Speaker 2: let's say it would allow you to feel more alert

762
00:40:19,280 --> 00:40:21,640
Speaker 2: or to sleep less, right, so kind of modulating some

763
00:40:22,120 --> 00:40:26,120
Speaker 2: circadian rhythms or energy level or attention. One could imagine

764
00:40:26,120 --> 00:40:28,799
Speaker 2: that that kind of like a performance enhancing drug that

765
00:40:28,840 --> 00:40:33,040
Speaker 2: could be done with a neurotechnology or neural interface. But

766
00:40:33,120 --> 00:40:35,680
Speaker 2: no one's done that yet in a way that's compelling.

767
00:40:36,560 --> 00:40:38,680
Speaker 2: People have talked about could it be kind of like

768
00:40:38,680 --> 00:40:41,279
Speaker 2: a coprocessor for your brain, like you know, somehow you

769
00:40:41,360 --> 00:40:45,400
Speaker 2: just know things. It's like having a smart AI assistant,

770
00:40:45,440 --> 00:40:48,040
Speaker 2: but it's inside your mind and it's much more seamless.

771
00:40:49,280 --> 00:40:51,040
Speaker 3: But that is a really long way away.

772
00:40:51,080 --> 00:40:53,640
Speaker 2: I mean, we have we're struggling to get you know,

773
00:40:54,040 --> 00:40:57,040
Speaker 2: crude vision in so people can can read a page. Now,

774
00:40:57,080 --> 00:40:59,759
Speaker 2: I mean, that's amazing, that's like very state of the art.

775
00:41:00,120 --> 00:41:04,160
Speaker 2: Or someone can slowly walk who has a spinal cord injury,

776
00:41:04,640 --> 00:41:08,680
Speaker 2: or someone can talk but not as eloquently as before

777
00:41:08,719 --> 00:41:11,200
Speaker 2: their als or before their stroke. So, given where we

778
00:41:11,239 --> 00:41:13,760
Speaker 2: are now, I think we're quite a ways away from

779
00:41:13,800 --> 00:41:15,640
Speaker 2: like beaming information in Oh.

780
00:41:15,719 --> 00:41:32,479
Speaker 1: I totally agree with you on that. I do wonder

781
00:41:32,560 --> 00:41:35,440
Speaker 1: twenty five years from now, let's say, right if you

782
00:41:35,560 --> 00:41:37,400
Speaker 1: just took a short cut of said, okay, look, I

783
00:41:37,440 --> 00:41:40,279
Speaker 1: want to listen to your covert speech things are not

784
00:41:40,320 --> 00:41:42,239
Speaker 1: saying out loud, and then I want to plug the

785
00:41:42,280 --> 00:41:44,719
Speaker 1: answer right back into your auditory cort text as though

786
00:41:44,760 --> 00:41:47,600
Speaker 1: you're hearing it, and then you know, beam wirelessly to

787
00:41:47,800 --> 00:41:50,719
Speaker 1: open AI or whatever exists in twenty five years from now. Yeah,

788
00:41:50,760 --> 00:41:53,480
Speaker 1: the question is could you ask a question and hear

789
00:41:53,520 --> 00:41:55,360
Speaker 1: the answer that way?

790
00:41:55,719 --> 00:41:58,880
Speaker 2: My prediction is yes, I think that could be done.

791
00:41:59,080 --> 00:42:00,319
Speaker 2: I mean also, I think that could be done the

792
00:42:00,360 --> 00:42:03,840
Speaker 2: next five years. It just would still require a surgery

793
00:42:04,040 --> 00:42:06,880
Speaker 2: to be done accurately, And so would anyone want it?

794
00:42:07,000 --> 00:42:10,600
Speaker 2: Would we as a society choose to allow? It?

795
00:42:10,600 --> 00:42:13,160
Speaker 3: Gets into debates of people's agency over their health.

796
00:42:13,320 --> 00:42:15,319
Speaker 1: Are there moral or ethical questions about that.

797
00:42:15,480 --> 00:42:18,759
Speaker 2: I think these are just general kind of medical and

798
00:42:18,840 --> 00:42:23,920
Speaker 2: societal questions of do we allow people to take medical

799
00:42:24,040 --> 00:42:27,560
Speaker 2: risks to get certain abilities that they otherwise wouldn't have.

800
00:42:28,120 --> 00:42:30,840
Speaker 1: One of the issues is about brain privacy, right, the

801
00:42:30,960 --> 00:42:34,640
Speaker 1: question of let's say I'm doing something that's recording my

802
00:42:34,880 --> 00:42:37,239
Speaker 1: covert thoughts, by which I mean, you know something that

803
00:42:37,280 --> 00:42:39,719
Speaker 1: I'm thinking, but I haven't actually pushed it out to

804
00:42:39,760 --> 00:42:43,080
Speaker 1: my motor cortex to say it yet. Who's the company

805
00:42:43,080 --> 00:42:48,520
Speaker 1: who has access to that? Do I want anybody accessing that?

806
00:42:49,080 --> 00:42:51,440
Speaker 2: I think that's yeah, that's a real concern. We're not

807
00:42:51,520 --> 00:42:54,400
Speaker 2: there yet, so to be clear, there's no BCI that

808
00:42:54,400 --> 00:42:56,960
Speaker 2: can decode covert thought yet exactly.

809
00:42:57,000 --> 00:42:59,839
Speaker 1: I'm talking twenty five years from Yeah. Yeah, I mean,

810
00:43:00,000 --> 00:43:03,080
Speaker 1: this is one of the conundrums about where this is heading.

811
00:43:03,440 --> 00:43:06,920
Speaker 2: Well, we're already dealing with inklings of that. So, for example,

812
00:43:06,960 --> 00:43:10,279
Speaker 2: in our system, because our participant is using this for

813
00:43:10,320 --> 00:43:12,520
Speaker 2: his day to day life. For example, one thing that

814
00:43:12,520 --> 00:43:15,600
Speaker 2: we implement was a privacy mode where if he toggles

815
00:43:15,600 --> 00:43:19,120
Speaker 2: a button, it no longer saves that data. This is

816
00:43:19,120 --> 00:43:22,239
Speaker 2: a academic clinical trial. In general, we're really loath to

817
00:43:22,239 --> 00:43:24,359
Speaker 2: give up any data I mean, it's so precious and

818
00:43:24,360 --> 00:43:28,359
Speaker 2: then these people are making these commitments to science, but

819
00:43:28,520 --> 00:43:30,239
Speaker 2: we also want to be respectful that he might need

820
00:43:30,280 --> 00:43:32,759
Speaker 2: to have a really private conversation and we don't want

821
00:43:32,800 --> 00:43:35,520
Speaker 2: to even have any ability to access that. So that's

822
00:43:35,560 --> 00:43:38,160
Speaker 2: already something we're dealing with in the context of a

823
00:43:38,239 --> 00:43:41,480
Speaker 2: medical trial from an academic medical center. I think this

824
00:43:41,520 --> 00:43:44,640
Speaker 2: is a very high trust scenario. Of course, when you

825
00:43:44,640 --> 00:43:47,200
Speaker 2: have companies that are building these, we're going to want

826
00:43:47,200 --> 00:43:49,360
Speaker 2: to think about we have what rights do in that

827
00:43:49,440 --> 00:43:53,080
Speaker 2: case patients or customers have to the data? Can the

828
00:43:53,160 --> 00:43:55,799
Speaker 2: data be used to improve the algorithms? Who owns the

829
00:43:55,840 --> 00:43:59,320
Speaker 2: benefit of that? What happens if a government subpoena?

830
00:43:59,360 --> 00:44:02,000
Speaker 3: Is it? Right? Now, we have.

831
00:44:02,000 --> 00:44:05,720
Speaker 2: This speech PCI for people with vocal tracked paralysis, meaning

832
00:44:05,760 --> 00:44:08,239
Speaker 2: that they know exactly what they're trying to say. The

833
00:44:08,280 --> 00:44:10,720
Speaker 2: words are clearly formed in their mind. They are trying

834
00:44:10,719 --> 00:44:14,880
Speaker 2: to speak it. Those commands are not reaching the muscles. Okay,

835
00:44:15,000 --> 00:44:18,520
Speaker 2: So we've shown that there is a very compelling therapy there.

836
00:44:19,120 --> 00:44:22,400
Speaker 2: Industry is going to come in and kind of productize it.

837
00:44:22,480 --> 00:44:24,480
Speaker 2: I think this is going to turn into medical device

838
00:44:24,840 --> 00:44:27,680
Speaker 2: in the next five years. There is a much larger

839
00:44:27,960 --> 00:44:31,920
Speaker 2: patient population though with aphasia due to stroke, So there

840
00:44:32,360 --> 00:44:35,360
Speaker 2: the problem is one step further upstream, meaning.

841
00:44:35,160 --> 00:44:36,799
Speaker 1: I mean they can't speak language by the way face.

842
00:44:36,960 --> 00:44:38,040
Speaker 3: Yes, well, there's different types.

843
00:44:38,080 --> 00:44:41,520
Speaker 2: So sometimes within aphasia that means they can't understand language,

844
00:44:41,560 --> 00:44:45,320
Speaker 2: but with expressive aphasia that means in many patients cases

845
00:44:45,440 --> 00:44:49,359
Speaker 2: they want to communicate, they really know what they're trying

846
00:44:49,400 --> 00:44:51,560
Speaker 2: to say in sort of in a meaning sense, but

847
00:44:51,640 --> 00:44:53,799
Speaker 2: they can't find the right words for it. It's almost like,

848
00:44:54,600 --> 00:44:57,000
Speaker 2: you know, sometimes I can't remember a word, but that's

849
00:44:57,120 --> 00:44:59,320
Speaker 2: rare and I can usually remember it or explain in

850
00:44:59,320 --> 00:45:02,160
Speaker 2: other words. But if I couldn't remember most of the words,

851
00:45:02,480 --> 00:45:04,520
Speaker 2: that would be really frustrating and debilitating.

852
00:45:04,520 --> 00:45:05,600
Speaker 3: And there's millions of.

853
00:45:05,520 --> 00:45:09,160
Speaker 2: People that have strokes and partially recover but never fully recover.

854
00:45:09,960 --> 00:45:12,880
Speaker 2: They have a language disorder. Many of them have perfectly

855
00:45:12,920 --> 00:45:17,200
Speaker 2: normal intelligence and their personalities preserved and kind of everything

856
00:45:17,200 --> 00:45:19,840
Speaker 2: else is there, but they just can't form words.

857
00:45:21,200 --> 00:45:22,000
Speaker 3: Can we help them?

858
00:45:22,040 --> 00:45:24,840
Speaker 2: And this is something that our lab and many others

859
00:45:24,880 --> 00:45:27,520
Speaker 2: are starting to think about. The idea is, can we

860
00:45:27,560 --> 00:45:30,160
Speaker 2: basically do this thing that we've done with a speech BCI,

861
00:45:30,239 --> 00:45:33,200
Speaker 2: but now make a language BCI can we put electrodes

862
00:45:33,600 --> 00:45:36,080
Speaker 2: somewhere in the language network and that is a lot

863
00:45:36,120 --> 00:45:38,359
Speaker 2: of the brain that's both a good and a bad thing.

864
00:45:39,239 --> 00:45:41,799
Speaker 3: Could we decode the meaning and this.

865
00:45:41,800 --> 00:45:43,439
Speaker 2: Is kind of getting close to this idea of a thought,

866
00:45:43,440 --> 00:45:45,799
Speaker 2: which is not a very well defined term, but could

867
00:45:45,800 --> 00:45:47,800
Speaker 2: we decode the semantic meaning of what they're trying to

868
00:45:47,840 --> 00:45:50,719
Speaker 2: communicate and have let's say, a tablet in front of

869
00:45:50,760 --> 00:45:53,680
Speaker 2: them print out a sentence or speak a sentence where

870
00:45:53,680 --> 00:45:56,320
Speaker 2: they're saying, I'm happy to see you, or could you

871
00:45:56,400 --> 00:45:59,319
Speaker 2: hand me some water? Or my nose itches or I'm

872
00:45:59,320 --> 00:46:02,880
Speaker 2: not feeling well well right, that thought, that communication intent

873
00:46:03,040 --> 00:46:06,440
Speaker 2: is still in there for many of these patients. We're

874
00:46:06,520 --> 00:46:10,120
Speaker 2: trying to develop a medical technology to help them, but

875
00:46:10,719 --> 00:46:13,400
Speaker 2: that starts getting pretty close to sounding like mind reading.

876
00:46:14,239 --> 00:46:17,960
Speaker 2: And so yeah, I think as an ethical question this

877
00:46:18,040 --> 00:46:22,279
Speaker 2: will potentially become relevant in the coming years if this

878
00:46:22,600 --> 00:46:24,120
Speaker 2: medical project succeeds.

879
00:46:24,360 --> 00:46:26,799
Speaker 1: It's interesting because we mean different things by mind reading.

880
00:46:26,840 --> 00:46:29,160
Speaker 1: There are all these different levels of it, so even

881
00:46:29,200 --> 00:46:33,400
Speaker 1: what somebody is trying to say often masks what they're thinking.

882
00:46:33,719 --> 00:46:36,480
Speaker 1: I'm trying to remember this quotation from the poet Oliver Goldsmith,

883
00:46:36,480 --> 00:46:39,279
Speaker 1: who said something like I think the real purpose of

884
00:46:39,400 --> 00:46:43,239
Speaker 1: language is not to communicate intent but to hide it.

885
00:46:44,480 --> 00:46:49,239
Speaker 1: So anyway, so if somebody says, hey, you know, I'm

886
00:46:49,239 --> 00:46:51,279
Speaker 1: happy to see you, or I you know, whatever the

887
00:46:51,320 --> 00:46:53,120
Speaker 1: thing is they're saying, it may or may not be

888
00:46:53,200 --> 00:46:55,960
Speaker 1: what their thoughts actually are. Is that's what their language is.

889
00:46:56,200 --> 00:46:59,040
Speaker 2: Yeah, so we're still talking. I'm still talking about decoding

890
00:46:59,040 --> 00:47:02,000
Speaker 2: communication and tent and that's sort of I think we

891
00:47:02,040 --> 00:47:04,440
Speaker 2: find it a little bit reassuring because it's an active process.

892
00:47:04,480 --> 00:47:08,120
Speaker 2: It's not like right now that we're nowhere close no

893
00:47:08,160 --> 00:47:09,680
Speaker 2: one even has an inkling of how to make a

894
00:47:09,680 --> 00:47:13,239
Speaker 2: device that can like read everything you know. You know,

895
00:47:13,239 --> 00:47:15,319
Speaker 2: you're not actively thinking about it, but it just knows

896
00:47:15,400 --> 00:47:18,160
Speaker 2: your whole childhood and all your deepest secrets and you

897
00:47:18,160 --> 00:47:21,040
Speaker 2: know what you think about everyone around you. That I

898
00:47:21,080 --> 00:47:22,880
Speaker 2: would not even know how to start to do that,

899
00:47:23,440 --> 00:47:26,879
Speaker 2: But for thinking what you're thinking actively or what you're

900
00:47:26,880 --> 00:47:31,560
Speaker 2: trying to communicate, that seems plausible. And there's some studies

901
00:47:31,680 --> 00:47:34,560
Speaker 2: using imaging that kind of you know, can do above

902
00:47:34,640 --> 00:47:37,520
Speaker 2: chance dey coding which someone's trying to communicate. We have

903
00:47:37,560 --> 00:47:39,879
Speaker 2: some preliminary data others do as well, So I think

904
00:47:40,160 --> 00:47:40,880
Speaker 2: that might happen.

905
00:47:41,080 --> 00:47:43,160
Speaker 1: So let me ask you a few things. When will

906
00:47:43,200 --> 00:47:44,880
Speaker 1: paralysis be solved?

907
00:47:44,960 --> 00:47:50,279
Speaker 2: I think there will be approved BCIs for paralysis in

908
00:47:50,360 --> 00:47:53,640
Speaker 2: about five years. That doesn't mean they'll be available everywhere.

909
00:47:53,960 --> 00:47:57,040
Speaker 2: They might be only available in certain markets. Maybe only

910
00:47:57,040 --> 00:48:00,200
Speaker 2: a few hospitals will initially be providing them, but that

911
00:48:00,200 --> 00:48:01,000
Speaker 2: will grow rapidly.

912
00:48:01,200 --> 00:48:01,960
Speaker 3: Will it mean.

913
00:48:01,920 --> 00:48:05,360
Speaker 2: Paralysis is cured? I think that's too strong a term.

914
00:48:06,080 --> 00:48:08,520
Speaker 2: Maybe that means you can walk slowly, you can move

915
00:48:08,560 --> 00:48:10,719
Speaker 2: your arm, but you maybe can't tie your shoelace.

916
00:48:10,800 --> 00:48:11,280
Speaker 3: Initially.

917
00:48:11,680 --> 00:48:14,239
Speaker 2: You can move a computer cursor really well, but that's

918
00:48:14,239 --> 00:48:15,720
Speaker 2: not the same thing as playing the piano.

919
00:48:16,120 --> 00:48:18,240
Speaker 3: So I think the capabilities will keep getting better.

920
00:48:18,600 --> 00:48:23,600
Speaker 1: And with als and dysarthria where someone can't articulate, well,

921
00:48:24,600 --> 00:48:25,680
Speaker 1: what are we looking at?

922
00:48:26,040 --> 00:48:28,240
Speaker 3: Your prediction, it's actually the same.

923
00:48:28,360 --> 00:48:32,439
Speaker 2: I think that the speech bring computer interfaces are going

924
00:48:32,480 --> 00:48:36,839
Speaker 2: to move very fast. I think that and cursor will

925
00:48:36,880 --> 00:48:39,239
Speaker 2: probably be one of the first approved systems, even though

926
00:48:39,239 --> 00:48:42,920
Speaker 2: people have been trying to move robot arms or paralyzed limbs.

927
00:48:42,760 --> 00:48:43,640
Speaker 3: For much longer.

928
00:48:43,880 --> 00:48:46,720
Speaker 2: So if you're trying to decode what someone's trying to say,

929
00:48:47,200 --> 00:48:49,600
Speaker 2: or decode them trying to move a computer cursor or

930
00:48:49,719 --> 00:48:52,200
Speaker 2: right of the keyboard the thing that they're trying to

931
00:48:52,200 --> 00:48:55,200
Speaker 2: control as a computer, and those are ubiquitous, they're everywhere, they're.

932
00:48:55,040 --> 00:48:56,480
Speaker 3: Cheap, they work really well.

933
00:48:56,760 --> 00:48:58,919
Speaker 2: If you're trying to decode what someone's trying to move

934
00:48:59,000 --> 00:49:02,400
Speaker 2: with their arm, you either need to move a robot arm.

935
00:49:02,680 --> 00:49:06,319
Speaker 2: Robot arms are hard, they break often, they're not as

936
00:49:06,480 --> 00:49:07,680
Speaker 2: precise as people are.

937
00:49:07,960 --> 00:49:09,520
Speaker 3: You know, where does it go? Does it go on

938
00:49:09,520 --> 00:49:10,239
Speaker 3: your wheelchair?

939
00:49:10,320 --> 00:49:13,000
Speaker 2: Is it there with you in the shower, if it's

940
00:49:13,040 --> 00:49:16,239
Speaker 2: mounted on like if you have an amputation, is.

941
00:49:16,160 --> 00:49:19,279
Speaker 3: It mounted on your stump or on your shoulder? That

942
00:49:19,400 --> 00:49:20,920
Speaker 3: is hard. There's a lot of challenges there.

943
00:49:22,080 --> 00:49:25,719
Speaker 2: So kind of the readout part for speech is very

944
00:49:25,719 --> 00:49:27,720
Speaker 2: hard because it's very fast. There's a lot of information

945
00:49:27,800 --> 00:49:31,839
Speaker 2: per second. But once you have that solved, making use

946
00:49:31,880 --> 00:49:33,840
Speaker 2: of it is actually really easy. You just send texts

947
00:49:33,840 --> 00:49:35,919
Speaker 2: to their computer or their phone, or you have their

948
00:49:36,200 --> 00:49:40,000
Speaker 2: tablet talk mix sound and that's something you can carry

949
00:49:40,000 --> 00:49:41,759
Speaker 2: with you all the time and it's really reliable. So

950
00:49:42,120 --> 00:49:44,080
Speaker 2: because for all those reasons, I think we're going to

951
00:49:44,120 --> 00:49:49,880
Speaker 2: have speech and also computer use BCIs hopefully starting to

952
00:49:49,960 --> 00:49:51,360
Speaker 2: hit the market in the next five years.

953
00:49:51,760 --> 00:49:54,440
Speaker 1: Great and when you think about fifty years from now,

954
00:49:54,480 --> 00:49:58,239
Speaker 1: when you think about as you're retiring and you look

955
00:49:58,280 --> 00:50:00,160
Speaker 1: around the field, what do you say.

956
00:50:00,880 --> 00:50:03,560
Speaker 2: I think BCIs will be well, the term may not

957
00:50:03,600 --> 00:50:06,120
Speaker 2: even mean anything because it's going to be so wide.

958
00:50:06,880 --> 00:50:09,640
Speaker 2: I think many of the diseases that we struggle with

959
00:50:09,680 --> 00:50:12,360
Speaker 2: today are going to be treated with some sort of

960
00:50:12,400 --> 00:50:15,040
Speaker 2: technology inside the head or interacting with the head.

961
00:50:15,120 --> 00:50:16,560
Speaker 3: Maybe it's somehow not.

962
00:50:16,600 --> 00:50:20,279
Speaker 2: Invasive, whether that's paralysis, which is going to be I

963
00:50:20,280 --> 00:50:24,240
Speaker 2: think much faster than that. Or will we have systems

964
00:50:24,239 --> 00:50:27,960
Speaker 2: that help us regulate our mood, Will they treat psychiatric issues,

965
00:50:28,040 --> 00:50:31,440
Speaker 2: Will they perhaps reconnect parts of the brain that have

966
00:50:31,520 --> 00:50:35,400
Speaker 2: been disconnected due to aging or damage, or injury or stroke.

967
00:50:36,200 --> 00:50:38,840
Speaker 2: If we're talking about fifty years, a lot can happen

968
00:50:38,840 --> 00:50:41,880
Speaker 2: in fifty years, right, I mean technology is moving very quickly.

969
00:50:42,480 --> 00:50:45,400
Speaker 2: The interfaces will get better. So instead of talking about

970
00:50:45,800 --> 00:50:47,960
Speaker 2: instead of me being right now excited about recording from

971
00:50:48,000 --> 00:50:51,560
Speaker 2: a thousand neurons, in fifty years, could we be interfacing

972
00:50:51,560 --> 00:50:53,600
Speaker 2: with one hundred thousand or a million neurons.

973
00:50:53,880 --> 00:50:55,160
Speaker 3: I think that's really plausible.

974
00:50:56,320 --> 00:51:01,719
Speaker 2: Through tiny nano wires or biohybrids or focused beams that

975
00:51:01,760 --> 00:51:02,600
Speaker 2: are non invasive.

976
00:51:02,840 --> 00:51:03,640
Speaker 3: A lot can happen.

977
00:51:03,640 --> 00:51:05,840
Speaker 2: In fifty years, our neuroscience, I think, will be a

978
00:51:05,840 --> 00:51:06,600
Speaker 2: lot more advanced.

979
00:51:06,800 --> 00:51:09,359
Speaker 3: We will not be limited to right now.

980
00:51:09,400 --> 00:51:12,480
Speaker 2: We mostly understand the peripheres, We understand movement, We understand

981
00:51:12,480 --> 00:51:15,840
Speaker 2: the senses really well because it's really easy to experimentally

982
00:51:15,960 --> 00:51:16,760
Speaker 2: manipulate those.

983
00:51:17,239 --> 00:51:18,279
Speaker 3: We as soon as you get.

984
00:51:18,160 --> 00:51:22,360
Speaker 2: Into the kind of the inside the center cognition intelligence,

985
00:51:22,400 --> 00:51:26,400
Speaker 2: how do we problem solve creativity? We don't understand that

986
00:51:26,440 --> 00:51:29,000
Speaker 2: really well, but I think at fifty years we will.

987
00:51:29,480 --> 00:51:31,640
Speaker 2: And part of that is because as we make these

988
00:51:31,840 --> 00:51:36,000
Speaker 2: medical systems, we will have access to human brains. So

989
00:51:36,200 --> 00:51:38,200
Speaker 2: think of this as a flywheel. So let's say someone

990
00:51:38,239 --> 00:51:40,880
Speaker 2: has a few thousand electrodes because they have a stroke

991
00:51:40,920 --> 00:51:44,000
Speaker 2: and they want to communicate. Maybe these are spread across

992
00:51:44,040 --> 00:51:46,360
Speaker 2: several different brain areas because you get different pieces of it.

993
00:51:46,480 --> 00:51:49,320
Speaker 2: Or maybe you get the prosody in one area primarily

994
00:51:49,400 --> 00:51:51,359
Speaker 2: and you get what they're trying to say in the

995
00:51:51,400 --> 00:51:54,960
Speaker 2: motor cortex. But you get some planning benefit and language

996
00:51:54,960 --> 00:51:56,960
Speaker 2: benefit from the temporal lobe. Okay, so let's say you

997
00:51:56,960 --> 00:52:00,879
Speaker 2: have four or five six areas that you're recording from. Well,

998
00:52:00,880 --> 00:52:02,920
Speaker 2: now you have a wealth of information that you can

999
00:52:03,000 --> 00:52:04,960
Speaker 2: use for other things. So some of these patients are

1000
00:52:04,960 --> 00:52:10,000
Speaker 2: going to develop dementia over time, or they might be depressed,

1001
00:52:10,440 --> 00:52:14,719
Speaker 2: or they might have OCD, And instead of having to

1002
00:52:14,800 --> 00:52:17,120
Speaker 2: do a new brain implant with all the new risks

1003
00:52:17,120 --> 00:52:18,600
Speaker 2: of that, you can just look at the data you're

1004
00:52:18,600 --> 00:52:21,360
Speaker 2: already collecting and try to relate that to their mood

1005
00:52:21,520 --> 00:52:23,719
Speaker 2: or what are they looking at? What are they trying

1006
00:52:23,719 --> 00:52:26,759
Speaker 2: to remember? Oh, they're trying to remember where they put

1007
00:52:26,800 --> 00:52:30,880
Speaker 2: their keys. Hey, Actually, because we have electrodes in the

1008
00:52:30,920 --> 00:52:34,040
Speaker 2: temporal lobe, it's close to the hippocampus, it's cortex, it's

1009
00:52:34,080 --> 00:52:36,400
Speaker 2: part of the memory system as well, everything's kind of

1010
00:52:36,440 --> 00:52:40,120
Speaker 2: spread out. Well, maybe now we're seeing some neural correlative

1011
00:52:40,400 --> 00:52:44,640
Speaker 2: that memory process. Maybe we can even ask if they're

1012
00:52:44,640 --> 00:52:48,080
Speaker 2: willing to do another clinical trail where we stimulate and

1013
00:52:48,520 --> 00:52:50,759
Speaker 2: try to boost that memory, try to kind of help

1014
00:52:50,840 --> 00:52:53,759
Speaker 2: nudget be remembered correctly. I think when we're talking about

1015
00:52:53,760 --> 00:52:56,520
Speaker 2: fifty years that's going to happen. And so through this

1016
00:52:56,560 --> 00:52:59,600
Speaker 2: process we're going to learn a lot more about how

1017
00:52:59,640 --> 00:53:01,680
Speaker 2: the human mind works and thus how to fix it.

1018
00:53:06,200 --> 00:53:09,360
Speaker 1: That was my interview with Sergei Stavisky, a neuroscientist that

1019
00:53:09,480 --> 00:53:13,400
Speaker 1: you see Davis and co director of the Neuroprosthetics Lab.

1020
00:53:13,840 --> 00:53:17,120
Speaker 1: We talked about what BCIs can do, what they might

1021
00:53:17,160 --> 00:53:20,759
Speaker 1: do soon, and how will navigate the human questions that

1022
00:53:20,800 --> 00:53:23,759
Speaker 1: they raise. What we talked about today was how a

1023
00:53:24,000 --> 00:53:28,480
Speaker 1: person's intention can find its way back into the world

1024
00:53:28,920 --> 00:53:33,240
Speaker 1: when bodies have lost function. Brain computer interfaces are opening

1025
00:53:33,320 --> 00:53:36,960
Speaker 1: a new lane right now. These technologies are crude in

1026
00:53:37,040 --> 00:53:39,759
Speaker 1: some ways, but they're getting better fast. Each year they

1027
00:53:39,760 --> 00:53:42,520
Speaker 1: get a little faster and more expressive. So this is

1028
00:53:42,600 --> 00:53:48,040
Speaker 1: how BCIs can restore autonomy and intimacy and dignity. And

1029
00:53:48,120 --> 00:53:51,279
Speaker 1: when it's done right, you don't see the technology at all,

1030
00:53:51,560 --> 00:53:54,520
Speaker 1: You just see the person again. So here's how I

1031
00:53:54,560 --> 00:53:57,440
Speaker 1: see it. In the next five years, BCIs are going

1032
00:53:57,520 --> 00:54:02,440
Speaker 1: to start looking less like research product and more like appliances.

1033
00:54:02,680 --> 00:54:06,560
Speaker 1: We're going to have fully implantable systems for communication. In

1034
00:54:06,600 --> 00:54:08,239
Speaker 1: other words, at some point in the future, we'll be

1035
00:54:08,280 --> 00:54:12,759
Speaker 1: looking at a small surgery, a wireless puck that goes in,

1036
00:54:13,239 --> 00:54:16,440
Speaker 1: and a setup that takes minutes instead of hours. You'll

1037
00:54:16,680 --> 00:54:20,480
Speaker 1: turn on your speech BCI or your BCI that controls

1038
00:54:20,480 --> 00:54:24,520
Speaker 1: a computer cursor, and the key thing will be reliability,

1039
00:54:24,880 --> 00:54:30,080
Speaker 1: these decoders will hold steady through years, and also identity.

1040
00:54:30,680 --> 00:54:33,760
Speaker 1: The voice is going to sound just like you, your cadence,

1041
00:54:33,840 --> 00:54:36,520
Speaker 1: your prosity, your humor at the end of a sentence.

1042
00:54:36,960 --> 00:54:40,880
Speaker 1: Maybe rehab teams will have a neural therapist who tunes

1043
00:54:40,920 --> 00:54:44,520
Speaker 1: your decoder the way that an audiologist tunes a cochlear implant.

1044
00:54:44,760 --> 00:54:46,720
Speaker 1: And if I had a guess, this will all become

1045
00:54:47,200 --> 00:54:52,319
Speaker 1: normal rather than newsworthy. Now around ten years out, we'll

1046
00:54:52,320 --> 00:54:56,080
Speaker 1: get good feedback of signals moving in both directions. So

1047
00:54:56,440 --> 00:55:00,360
Speaker 1: a person who is suffering from paralysis will can control

1048
00:55:00,400 --> 00:55:04,400
Speaker 1: her hand through say electrodes in her motor cortex, and

1049
00:55:04,440 --> 00:55:08,080
Speaker 1: you have another interface, say electrodes in her somatosentury cortex,

1050
00:55:08,520 --> 00:55:12,520
Speaker 1: that's inputting information so that she feels a push back

1051
00:55:12,640 --> 00:55:17,280
Speaker 1: with electrically evoked touch, and that loop makes the movements

1052
00:55:17,640 --> 00:55:20,920
Speaker 1: smooth and automatic. This is all going to continue getting

1053
00:55:20,960 --> 00:55:25,160
Speaker 1: smaller and better. Soon will have thin film options to

1054
00:55:25,280 --> 00:55:30,399
Speaker 1: reduce the surgical footprints. The decoders will auto calibrate, they'll

1055
00:55:30,440 --> 00:55:34,640
Speaker 1: borrow tricks from language models, and they'll figure out how

1056
00:55:34,680 --> 00:55:38,759
Speaker 1: to adjust to your neural dynamics when you're tired or

1057
00:55:38,800 --> 00:55:43,760
Speaker 1: stressed or boosted on caffeine. Eventually your BCI will speak

1058
00:55:43,800 --> 00:55:47,839
Speaker 1: the same API language as your phone and home devices,

1059
00:55:48,000 --> 00:55:51,040
Speaker 1: so that you can text or adjust the lights or

1060
00:55:51,080 --> 00:55:56,160
Speaker 1: turn on appliances without moving a limb or making a sound.

1061
00:55:56,440 --> 00:56:02,560
Speaker 1: And crucially, the privacy architecture is to evolve like inner

1062
00:56:02,560 --> 00:56:06,960
Speaker 1: speech stays off limits by default, and your neural stream

1063
00:56:07,080 --> 00:56:10,799
Speaker 1: lives behind consent gates. We'll need to have a kind

1064
00:56:10,840 --> 00:56:14,960
Speaker 1: of airplane mode for the mind. Okay, And if I

1065
00:56:15,000 --> 00:56:18,120
Speaker 1: were going to speculate on a quarter century from now,

1066
00:56:18,640 --> 00:56:21,600
Speaker 1: I'm thinking that what we're looking at is very high

1067
00:56:21,640 --> 00:56:26,040
Speaker 1: bandwidth arrays. These might be micro needles or flexible meshes,

1068
00:56:26,600 --> 00:56:31,200
Speaker 1: or electrode stents living on the inside of the blood vessels.

1069
00:56:31,480 --> 00:56:34,920
Speaker 1: Whatever the technology, it's going to give us coverage that

1070
00:56:35,080 --> 00:56:41,120
Speaker 1: approaches the dexterousness of natural hand control. Imagine playing a

1071
00:56:41,160 --> 00:56:45,880
Speaker 1: piano with one of these. Imagine prosthetics and exoskeletons that

1072
00:56:46,000 --> 00:56:49,840
Speaker 1: feel less like machines and more like natural limbs because

1073
00:56:49,880 --> 00:56:53,399
Speaker 1: the brain sees and feels them just as part of

1074
00:56:53,480 --> 00:56:57,280
Speaker 1: the body. And for communication, we'll get the full richness

1075
00:56:57,320 --> 00:57:00,759
Speaker 1: of natural speech. Just imagine talking with a person with

1076
00:57:00,800 --> 00:57:04,680
Speaker 1: a BCI and you hear the emphasis of ups and

1077
00:57:04,719 --> 00:57:08,560
Speaker 1: downs of speech, and their laughter and their little half

1078
00:57:08,600 --> 00:57:14,200
Speaker 1: swallowed syllables when people are negotiating, turn taking and singing.

1079
00:57:15,080 --> 00:57:17,560
Speaker 1: And soon enough, I think, in our lifetimes for sure,

1080
00:57:18,080 --> 00:57:21,640
Speaker 1: the science fiction edge of this all is going to

1081
00:57:21,680 --> 00:57:24,680
Speaker 1: start to glow. So imagine a scene like this when

1082
00:57:24,720 --> 00:57:27,640
Speaker 1: you step onto a train maybe thirty five years from now.

1083
00:57:28,160 --> 00:57:32,240
Speaker 1: People are sitting there. It's crowded, and they're all speaking

1084
00:57:32,440 --> 00:57:36,040
Speaker 1: private messages to their friends who are somewhere else. There's

1085
00:57:36,080 --> 00:57:40,920
Speaker 1: no sound, the train is quiet. Each person's decoder is

1086
00:57:41,040 --> 00:57:45,120
Speaker 1: locked onto their attempted speech, not their idle thoughts, and

1087
00:57:45,280 --> 00:57:48,960
Speaker 1: every message is signed with a cryptographic water mark that

1088
00:57:49,040 --> 00:57:52,520
Speaker 1: proves it came from that person's neural key. So you're

1089
00:57:52,720 --> 00:57:57,160
Speaker 1: looking at a silent train car, but it's filled with conversations.

1090
00:57:57,640 --> 00:58:01,960
Speaker 1: Or just imagine something simpler. Here's a carpenter who lost

1091
00:58:01,960 --> 00:58:04,960
Speaker 1: his hand, but he's back at work with a prosthetic

1092
00:58:05,000 --> 00:58:10,680
Speaker 1: hand that streams touch information into the brain pressure and temperature.

1093
00:58:10,840 --> 00:58:13,960
Speaker 1: But also he can feel the details of the grain.

1094
00:58:14,080 --> 00:58:17,280
Speaker 1: He can tell the difference between pine and oak just

1095
00:58:17,320 --> 00:58:21,800
Speaker 1: by running his sensory packed robotic fingers over it. And

1096
00:58:21,840 --> 00:58:24,280
Speaker 1: the key is that He doesn't think about the device

1097
00:58:24,440 --> 00:58:28,560
Speaker 1: at all. He just builds, just like you use the

1098
00:58:29,120 --> 00:58:33,040
Speaker 1: high bandwidth sensory devices on your own hand, and you

1099
00:58:33,160 --> 00:58:36,760
Speaker 1: rarely stop to think about it. Eventually, there'll be a

1100
00:58:36,800 --> 00:58:39,640
Speaker 1: lot of legislation in place, because there are going to

1101
00:58:39,680 --> 00:58:43,240
Speaker 1: be hard lines we choose as a society not to cross.

1102
00:58:43,680 --> 00:58:47,080
Speaker 1: Not all thoughts should be digitized. We're going to need

1103
00:58:47,480 --> 00:58:51,520
Speaker 1: neuro rights with teeth, will need on device processing that

1104
00:58:51,640 --> 00:58:55,840
Speaker 1: keeps data local where maybe you have your own descendant

1105
00:58:55,960 --> 00:58:59,920
Speaker 1: of modern day LLMS living with you in your brain.

1106
00:59:00,680 --> 00:59:05,640
Speaker 1: Whatever the case, will presumably keep asking philosophical questions about

1107
00:59:05,680 --> 00:59:09,280
Speaker 1: our brains and ourselves, but we'll get to do it

1108
00:59:09,320 --> 00:59:13,400
Speaker 1: with better and better tools than we have now. And

1109
00:59:13,560 --> 00:59:17,200
Speaker 1: I think what this means is that we have more

1110
00:59:17,280 --> 00:59:21,440
Speaker 1: in common with our ancestors of a thousand years ago

1111
00:59:22,000 --> 00:59:28,760
Speaker 1: than we do with our descendants a century from now.

1112
00:59:29,880 --> 00:59:32,840
Speaker 1: Go to Eagleman dot com slash podcast for more information

1113
00:59:32,920 --> 00:59:36,960
Speaker 1: and to find further reading. Send me an email at

1114
00:59:37,040 --> 00:59:41,320
Speaker 1: podcasts at eagleman dot com with questions or discussion and

1115
00:59:41,400 --> 00:59:44,360
Speaker 1: check out Subscribe to Inner Cosmos on YouTube for videos

1116
00:59:44,400 --> 00:59:48,880
Speaker 1: of each episode and to leave comments. Until next time.

1117
00:59:48,960 --> 01:00:03,480
Speaker 1: I'm David eagleman, and this is inner cosmos.