WEBVTT - Ep12 "Can we create new senses for humans?" 0:00:05.000 --> 0:00:08.720 Can a person who is blind come to see through 0:00:08.800 --> 0:00:13.080 her tongue? Can a baby be born without ears? What 0:00:13.119 --> 0:00:15.680 does it like to have the smell of a dog? 0:00:15.840 --> 0:00:17.680 And what does any of this have to do with 0:00:17.920 --> 0:00:25.080 airplane pilots or Westworld or potato head. Welcome to Inner 0:00:25.160 --> 0:00:29.520 Cosmos with me, David Eagleman. I'm a neuroscientist and an author, 0:00:30.000 --> 0:00:32.480 and my fascination for a very long time has been 0:00:32.640 --> 0:00:37.599 how brains perceive reality, because the strange part is that 0:00:37.600 --> 0:00:40.199 we're not seeing most of the action that's going on 0:00:40.400 --> 0:00:43.279 out there. So today we're going to dive into that 0:00:43.440 --> 0:00:47.600 and we're going to see how we might expand our perception. 0:00:50.560 --> 0:00:54.080 We're built out of really small stuff like DNA, and 0:00:54.160 --> 0:00:58.320 we're embedded in a very large cosmos, and we're not 0:00:58.480 --> 0:01:03.480 particularly good at receiving reality at either of those scales. 0:01:03.880 --> 0:01:06.360 And that's because we've evolved to deal with reality. It's 0:01:06.360 --> 0:01:09.360 it's very thin slice in between at the level of 0:01:10.120 --> 0:01:12.840 rivers and apples and rabbits and stuff like that. But 0:01:13.160 --> 0:01:17.240 even here, at our level of perception, we're not seeing 0:01:17.800 --> 0:01:21.240 most of the action that's going on. So take, for example, 0:01:21.319 --> 0:01:24.240 the colors of our world, So picture the reds and 0:01:24.319 --> 0:01:28.000 blues and greens and purples. These are light waves that 0:01:28.200 --> 0:01:31.800 bounce off objects and hit these specialized receptors at the 0:01:31.800 --> 0:01:35.039 back of our eyes, and then we perceive these colors, 0:01:35.560 --> 0:01:38.400 but we're not seeing all the light waves that are 0:01:38.440 --> 0:01:42.200 out there. In fact, what we see is less than 0:01:42.240 --> 0:01:46.160 a ten trillionth of the light waves out there. So 0:01:46.280 --> 0:01:49.160 if you look at what's called the electromagnetic spectrum, you 0:01:49.280 --> 0:01:55.160 have radio waves and microwaves and X rays and gamma rays. 0:01:55.280 --> 0:01:58.640 All these are light. They're just different frequencies. These are 0:01:58.840 --> 0:02:02.640 passing through your body right now, and you're completely unaware 0:02:02.680 --> 0:02:06.240 of them because your biology doesn't come with the right 0:02:06.320 --> 0:02:10.960 receptors to pick those up. They are light, but they're 0:02:10.960 --> 0:02:15.480 not visible light. There are thousands of cell phone conversations 0:02:15.520 --> 0:02:20.160 passing through you right now, and you're completely blind to them. Now, 0:02:20.200 --> 0:02:25.280 it's not that these other wavelengths of light are inherently unseeable. 0:02:25.840 --> 0:02:30.560 Snakes include some infrared light in their reality, and honeybees 0:02:30.600 --> 0:02:34.400 include ultraviolet light in their view of the world. And 0:02:34.440 --> 0:02:37.760 of course we build machines in the dashboards of our 0:02:37.800 --> 0:02:41.079 cars to pick up on signals in the radio frequency range, 0:02:41.440 --> 0:02:43.960 and we build machines and hospitals to pick up on 0:02:44.040 --> 0:02:47.400 the X ray range and so on. But you can't 0:02:47.440 --> 0:02:50.360 sense any of these things by yourself, at least not yet, 0:02:50.720 --> 0:02:55.480 because you don't come equipped with the proper sensors. Now, 0:02:55.520 --> 0:02:59.320 what this means is that our experience of reality is 0:03:00.000 --> 0:03:03.760 it's trained by our biology, and that goes against this 0:03:03.880 --> 0:03:07.000 common sense notion that our eyes and our ears and 0:03:07.040 --> 0:03:10.880 our fingertips are just picking up on the objective reality 0:03:10.919 --> 0:03:14.639 out there. Instead, what this means is that our brains 0:03:14.680 --> 0:03:19.880 are sampling just a little bit of the world. Now, 0:03:19.919 --> 0:03:24.480 across the animal kingdom, different animals pick up on different 0:03:24.720 --> 0:03:29.480 parts of reality. So take the tick. It's blind and 0:03:29.560 --> 0:03:33.399 death and in its little world, the important signals are 0:03:33.840 --> 0:03:38.240 temperature and body odor butteric acid, and that's all it 0:03:38.320 --> 0:03:41.480 picks up on, and that's how it constructs its reality. 0:03:42.080 --> 0:03:46.640 For a fish called the black ghost knife fish, its 0:03:46.640 --> 0:03:51.560 sensory world is all about electrical fields and the perturbations 0:03:51.600 --> 0:03:54.600 of those fields when it's passing a rock or another creature, 0:03:55.040 --> 0:03:59.640 and that's always picking up on. For the echolocating bat, 0:04:00.160 --> 0:04:03.600 its reality is constructed out of air compression waves that 0:04:03.680 --> 0:04:06.520 bounce off objects and come back to them. So for 0:04:06.600 --> 0:04:10.880 these different animals. That's the slice of their ecosystem that 0:04:10.880 --> 0:04:13.720 they can pick up on, and that's all they're seeing. 0:04:13.720 --> 0:04:16.280 And we have a word for this in science. This 0:04:16.320 --> 0:04:19.760 is called the umveldt, which is the German word for 0:04:19.839 --> 0:04:24.359 the surrounding world. Now, every animal is very limited in 0:04:24.400 --> 0:04:27.520 the umveldt that it can pick up on, but presumably 0:04:27.600 --> 0:04:32.080 every animal assumes that it's umveldt is the entire objective 0:04:32.160 --> 0:04:35.000 reality that's out there, because why would you ever stop 0:04:35.040 --> 0:04:39.599 to imagine that there's something beyond what you can sense. Instead, 0:04:39.680 --> 0:04:43.440 we all accept reality as it is presented to us. 0:04:43.920 --> 0:04:48.080 So let's do a consciousness raiser on this. Imagine that 0:04:48.279 --> 0:04:53.160 you are your family dog, and your whole world is 0:04:53.160 --> 0:04:56.400 about smelling. So you've got this long snout that has 0:04:56.440 --> 0:04:59.360 two hundred million scent receptors in it, and you have 0:04:59.480 --> 0:05:04.160 wet nostrils that attract and trap scent molecules. And your 0:05:04.160 --> 0:05:06.800 nostrils even have slits so you can take these big 0:05:06.839 --> 0:05:09.920 nose fulls of air. You have floppy ears to kick 0:05:10.000 --> 0:05:14.080 up more scent. Everything is about smell for you. So 0:05:14.200 --> 0:05:18.520 one day you stop in your tracks with a revelation 0:05:18.720 --> 0:05:21.839 and you look at your human owners and you think, 0:05:22.279 --> 0:05:25.360 what is it like to have the pitiful little nose 0:05:25.520 --> 0:05:27.720 of a human? What is it like when they take 0:05:27.920 --> 0:05:31.800 a little feeble nose full of air? How can a 0:05:31.880 --> 0:05:34.640 human not know that there's a cat one hundred yards 0:05:34.680 --> 0:05:38.120 away or that their best friend was on this very 0:05:38.200 --> 0:05:41.760 spot six hours ago. But because we're humans that we've 0:05:41.800 --> 0:05:46.440 never experienced that world of smell, we don't miss it 0:05:46.480 --> 0:05:48.839 and we don't even think about it. Because we are 0:05:48.920 --> 0:05:51.920 firmly settled into our umvelt we don't feel like there's 0:05:51.960 --> 0:05:55.600 a black hole of smell that we're missing there. We 0:05:55.640 --> 0:05:58.640 think we've got the whole world. But the question is 0:05:58.760 --> 0:06:02.320 do we have to be stuck in the umveldt into 0:06:02.360 --> 0:06:05.839 which we were born? So as a neuroscientist, I've always 0:06:05.880 --> 0:06:09.520 been interested in the way that our technology might allow 0:06:09.600 --> 0:06:12.839 us to expand our umveldt and how that's going to 0:06:12.920 --> 0:06:17.560 change the experience of being human. So we're already quite 0:06:17.600 --> 0:06:22.560 good at marrying our technology to our biology. You may 0:06:22.640 --> 0:06:25.599 know this, but there are hundreds of thousands of people 0:06:25.680 --> 0:06:30.400 walking around with artificial hearing and artificial vision. The way 0:06:30.440 --> 0:06:33.599 this works, for example, with artificial hearing is you have 0:06:34.040 --> 0:06:37.680 a microphone and you digitize the signal and you put 0:06:37.720 --> 0:06:42.160 an electrode strip directly into the inner ear. Or with 0:06:42.360 --> 0:06:45.640 artificial vision, you have what's called a retinal implant, where 0:06:45.680 --> 0:06:49.400 you take a camera and you digitize this signal and 0:06:49.440 --> 0:06:53.560 you plug an electrode grid directly into the back of 0:06:53.600 --> 0:06:57.120 the eye and the optic nerve. Now, as recently as 0:06:57.320 --> 0:06:59.719 twenty five years ago, there were a lot of scientists 0:06:59.720 --> 0:07:03.640 who these technologies were never going to work. Why. It's 0:07:03.680 --> 0:07:07.279 because these technologies speak the language of silicon valley and 0:07:07.400 --> 0:07:10.880 zeros and ones, and it's not exactly the same dialect 0:07:11.000 --> 0:07:16.280 as our natural biological sense organs. But the fact is 0:07:16.520 --> 0:07:19.920 that these technologies work. The brain figures out how to 0:07:20.040 --> 0:07:23.240 use the signals just fine. Now, how do we understand that? 0:07:24.720 --> 0:07:28.320 The key to understanding this requires diving one level deeper. 0:07:29.040 --> 0:07:33.360 Your three pounds of brain tissue are not hearing or 0:07:33.440 --> 0:07:37.120 seeing the world around you directly. It's not that your 0:07:37.160 --> 0:07:40.360 eyes are piping in light or your ears are piping 0:07:40.520 --> 0:07:45.760 sound in. Instead, your brain is locked in a crypt 0:07:46.000 --> 0:07:51.680 of silence and darkness inside your skull. All inever experiences 0:07:52.080 --> 0:07:57.320 are electrochemical signals that stream in along different data cables. 0:07:57.360 --> 0:07:59.600 That's all it has to work with are these little 0:07:59.800 --> 0:08:05.080 electrical spikes and chemical releases. It's just a world of 0:08:05.120 --> 0:08:09.240 spikes running around in darkness inside there, and in ways 0:08:09.280 --> 0:08:12.200 that we're still working to understand. The brain is shockingly 0:08:12.320 --> 0:08:16.680 good at taking these signals running around and extracting patterns 0:08:16.720 --> 0:08:20.680 into those patterns that as signs meaning, and with that meaning, 0:08:20.720 --> 0:08:26.160 you have subjective experience. So the brain is an organ 0:08:26.560 --> 0:08:30.160 that converts sparks in the dark into a picture show 0:08:30.360 --> 0:08:35.000 of your world. All the hues and aromas and emotions 0:08:35.040 --> 0:08:39.040 and sensations of your life. These are encoded in trillions 0:08:39.040 --> 0:08:42.720 of signals zipping around in the blackness. So you know 0:08:42.760 --> 0:08:45.800 when you watch a beautiful screensaver on your computer screen, 0:08:46.160 --> 0:08:49.120 that's just built out of zeros and ones and transistors, 0:08:49.840 --> 0:08:52.600 and it's somehow the same thing that's happening with your 0:08:52.760 --> 0:08:56.360 experience of the world. Let's understand this just a little 0:08:56.400 --> 0:08:59.640 bit more. Imagine that you traveled over to an island 0:08:59.760 --> 0:09:03.000 of people who are all born blind, so they all 0:09:03.080 --> 0:09:07.400 read by braille. They feel tiny patterns of inputs on 0:09:07.440 --> 0:09:11.000 their fingertips. So you watch them read a book and 0:09:11.040 --> 0:09:14.520 they're brushing over the small bumps with their fingers and 0:09:14.559 --> 0:09:18.400 you watch them laugh and cry at the book they're reading, 0:09:18.440 --> 0:09:21.960 and you might wonder how can they fit all that 0:09:22.400 --> 0:09:25.760 emotion into the tip of their finger. So you explain 0:09:25.800 --> 0:09:28.760 to them that when you read a novel, you aim 0:09:28.880 --> 0:09:34.000 these spheres on your face towards visual patterns of lines 0:09:34.040 --> 0:09:37.200 and curves on a page, and each of your eyes 0:09:37.280 --> 0:09:41.720 has a lawn of cells that catch photons, and in 0:09:41.760 --> 0:09:45.520 this way you can register the shapes of the symbols. 0:09:45.559 --> 0:09:47.960 And you tell them that you have memorized a set 0:09:47.960 --> 0:09:52.160 of rules by which different shapes on the page represent 0:09:52.320 --> 0:09:55.920 different sounds. So for each squiggle that you detect with 0:09:56.040 --> 0:09:59.320 your eyes, you recite a small sound in your head, 0:10:00.000 --> 0:10:02.760 imagining what you would hear if someone were speaking that 0:10:02.840 --> 0:10:07.800 out loud. And so the resulting pattern of neurochemical signaling 0:10:08.120 --> 0:10:12.920 makes you laugh or cry. You couldn't blame the islanders 0:10:12.960 --> 0:10:17.360 for finding your story difficult to understand. How do you 0:10:17.400 --> 0:10:21.120 fit all that emotion into two spheres on your head? Okay, 0:10:21.320 --> 0:10:24.679 So you or they would finally have to allow something, 0:10:24.760 --> 0:10:28.920 which is that the fingertip or the eyeball is just 0:10:29.040 --> 0:10:33.360 the peripheral device that converts information from the outside world 0:10:33.800 --> 0:10:37.160 into spikes in the brain. And then the brain does 0:10:37.200 --> 0:10:41.280 all the hard work of the interpretation. You and the 0:10:41.280 --> 0:10:44.760 Islanders would break bread over the fact that in the end, 0:10:44.800 --> 0:10:48.240 it's all about the trillions of spikes racing around in 0:10:48.280 --> 0:10:52.040 the brain, and that the method of entry simply isn't 0:10:52.040 --> 0:10:55.600 the part that matters, because your brain doesn't know and 0:10:55.640 --> 0:10:59.200 it doesn't care where it gets the data from. Whatever 0:10:59.240 --> 0:11:03.200 information comes in from the outside, it just figures out 0:11:03.360 --> 0:11:06.160 what to do with it. And this is a very 0:11:06.280 --> 0:11:11.360 efficient kind of machine. It is essentially a general purpose 0:11:11.440 --> 0:11:15.080 computing device. It just takes in everything and it figures 0:11:15.080 --> 0:11:17.480 out what it's going to do with it. And in 0:11:17.520 --> 0:11:21.200 my work, I've proposed that this freeze up mother nature 0:11:21.640 --> 0:11:26.720 to tinker around with different sorts of input channels. So 0:11:26.800 --> 0:11:29.520 I've argued in my talks and books and papers that 0:11:29.559 --> 0:11:35.000 we can send information into the brain via unusual pathways. 0:11:35.760 --> 0:11:39.480 And I call this the pH model of evolution. And 0:11:39.480 --> 0:11:41.600 I don't want to get too technical here, but pH 0:11:41.640 --> 0:11:44.760 stands for potato head, and I use this name to 0:11:44.840 --> 0:11:49.079 emphasize that all these sensors that we know in love, 0:11:49.360 --> 0:11:53.960 like our eyes and our ears and our fingertips, these 0:11:54.040 --> 0:11:59.160 are merely peripheral, plug and play devices, you stick them 0:11:59.160 --> 0:12:01.840 in and you're good to go, just like with a 0:12:01.880 --> 0:12:06.319 potato head. Where you attach these devices, the brain figures 0:12:06.360 --> 0:12:09.080 out what to do with the data that comes in. 0:12:09.679 --> 0:12:12.559 And by the way, when you look across the animal kingdom, 0:12:12.840 --> 0:12:17.680 you find lots of interesting peripheral devices. So snakes have 0:12:18.080 --> 0:12:22.040 heat pits with which they detect the infrared light. And 0:12:22.120 --> 0:12:26.960 the black ghost knifefish has electro receptors up and down 0:12:27.000 --> 0:12:29.280 its body. That's how it detects the changes in the 0:12:29.360 --> 0:12:33.160 electrical field. And there's an animal called the star nosed mole, 0:12:33.200 --> 0:12:37.320 which essentially has this nose with twenty two fingers on it, 0:12:37.640 --> 0:12:41.120 and it moves around through its three dimensional tunnel system 0:12:41.160 --> 0:12:44.800 and feels around and constructs a model of its world 0:12:44.840 --> 0:12:49.880 that way. And many birds and cows and insects have 0:12:49.960 --> 0:12:54.240 specializations so that they can feel the magnetic field of 0:12:54.320 --> 0:12:57.640 the planet. This is called magneto reception, and they navigate 0:12:57.720 --> 0:13:02.640 that way. The idea with the potato head model is 0:13:02.679 --> 0:13:07.720 that Mother Nature doesn't have to continually redesign the brain 0:13:07.920 --> 0:13:12.800 every time she introduces some new peripheral device. Instead, with 0:13:12.920 --> 0:13:17.439 the principles of brain operation already established, all she has 0:13:17.520 --> 0:13:21.120 to do is worry about designing new peripheral devices to 0:13:21.160 --> 0:13:24.600 pick up on new information from the world. So in 0:13:24.640 --> 0:13:27.480 the same way you can plug an arbitrary nose or 0:13:27.520 --> 0:13:32.280 eyes or mouth into potato head. Likewise, nature plugs all 0:13:32.360 --> 0:13:36.199 kinds of instrumentation into the brain for the purpose of 0:13:36.280 --> 0:13:54.760 detecting these energy sources in the outside world. Now, the 0:13:54.800 --> 0:14:00.679 idea of looking at our peripheral sensors like individuals standalone 0:14:00.679 --> 0:14:05.720 devices might seem bizarre, because, after all, aren't there thousands 0:14:05.800 --> 0:14:08.960 of genes involved with building these devices, and don't these 0:14:09.040 --> 0:14:12.319 genes overlap with other pieces and parts of the body. 0:14:12.640 --> 0:14:16.040 Can we really look at the nose or the eye, 0:14:16.120 --> 0:14:18.800 or the ear or the tongue as a device that 0:14:18.920 --> 0:14:22.280 stands alone. So I started studying this question because I thought, 0:14:22.560 --> 0:14:26.440 if the potato head model is correct, wouldn't that suggest 0:14:26.520 --> 0:14:30.200 we might find switches in the genetics that lead to 0:14:30.920 --> 0:14:34.360 the presence or absence of a peripheral device. And as 0:14:34.400 --> 0:14:39.400 it turns out, that's precisely what can happen. So, for example, 0:14:39.560 --> 0:14:43.800 some babies are born completely missing a nose, and they 0:14:43.840 --> 0:14:48.240 also lack the nasal cavity and the whole system for smelling. 0:14:48.640 --> 0:14:51.720 This is called a rhinia. Now, these kind of mutations, 0:14:51.720 --> 0:14:55.240 they seem startling and difficult to fathom, But in our 0:14:55.360 --> 0:14:59.360 plug and play framework, a rhinia is predictable. With a 0:14:59.440 --> 0:15:03.800 slight tweak of the genes, the peripheral device simply doesn't 0:15:03.800 --> 0:15:07.080 get built. Or consider other babies who are born normal, 0:15:07.160 --> 0:15:11.440 but they have no eyes. This is called anophthalmia, and 0:15:11.560 --> 0:15:16.320 others are born without tongues. Some babies are born without ears, 0:15:16.400 --> 0:15:21.360 that's called anotia. Some children are born without any pain receptors, 0:15:21.720 --> 0:15:24.600 and more generally, others are born without any touch receptors. 0:15:24.600 --> 0:15:27.680 This is called nafia. And so when we look at 0:15:27.680 --> 0:15:33.520 these situations, it becomes clear that our peripheral detectors unpack 0:15:33.680 --> 0:15:37.120 because of specific genetic programs. And if you have a 0:15:37.200 --> 0:15:41.400 minor mauthfunction in the genes, that can halt the program, 0:15:41.720 --> 0:15:45.040 and then the brain just doesn't get that particular data 0:15:45.040 --> 0:15:49.160 stream of information from the world, whether that's smell, molecules, 0:15:49.280 --> 0:15:52.400 or photons or air compression waves or touch or whatever. 0:15:53.120 --> 0:15:55.760 For me, the lesson that comes together here is that 0:15:55.880 --> 0:16:00.240 nature designs ways of extracting information from the world world, 0:16:00.560 --> 0:16:04.440 and these unpack with their own little genetic instructions. Now, 0:16:04.480 --> 0:16:09.600 what this implies is that there's nothing really fundamental about 0:16:09.600 --> 0:16:12.200 the devices that you and I come to the table 0:16:12.240 --> 0:16:14.440 with our eyes and our ears and our nose and 0:16:14.480 --> 0:16:18.760 our fingertips. It's just what we've inherited from a complex 0:16:18.920 --> 0:16:23.960 road of evolution. But that particular collection of sensors might 0:16:24.000 --> 0:16:27.200 not have to be what we stick with, because the 0:16:27.280 --> 0:16:32.640 brain's ability to decode different kinds of incoming information implies 0:16:32.720 --> 0:16:35.400 the crazy prediction that you might be able to get 0:16:35.720 --> 0:16:39.360 some sensory cable going into the brain to carry a 0:16:39.440 --> 0:16:42.920 different kind of sensory information. For example, what if you 0:16:43.000 --> 0:16:47.160 took a data stream from a video camera and converted 0:16:47.200 --> 0:16:51.280 that into touch on your skin. Would the brain eventually 0:16:51.320 --> 0:16:56.320 be able to interpret the visual world simply by feeling it? 0:16:56.680 --> 0:17:02.720 And this is the stranger than fiction world of sensory substitution. 0:17:03.880 --> 0:17:08.520 Sensory substitution refers to the idea of feeding information into 0:17:08.600 --> 0:17:12.800 the brain via unusual sensory channels and the brain just 0:17:12.880 --> 0:17:15.520 figures out what to do with the information. Now, that 0:17:15.600 --> 0:17:19.919 might sound speculative, but the first paper demonstrating this was 0:17:19.960 --> 0:17:24.200 published in the journal Nature in nineteen sixty nine. There 0:17:24.240 --> 0:17:28.280 was a scientist named Paul Baki Rita and he put 0:17:28.359 --> 0:17:32.640 blind people in a modified dental chair and he set 0:17:32.720 --> 0:17:35.080 up a video feed and he would put something in 0:17:35.080 --> 0:17:37.879 front of the camera and then the person would feel 0:17:37.920 --> 0:17:42.000 that poked into their back with a grid of solenoids. 0:17:42.200 --> 0:17:44.159 So if he put a coffee cup in front of 0:17:44.160 --> 0:17:47.160 the camera, they would feel the shape of a coffee 0:17:47.200 --> 0:17:49.760 cup in their back. Or he puts a telephone in 0:17:49.760 --> 0:17:52.199 front of the camera, and they feel a telephone in 0:17:52.240 --> 0:17:56.560 their back. And amazingly, people who were blind got pretty 0:17:56.600 --> 0:17:59.800 good at being able to determine what was in front 0:17:59.840 --> 0:18:02.920 of them the camera, just by feeling it in the 0:18:02.960 --> 0:18:07.359 small of their back. So Baki Rita summarized his findings 0:18:07.400 --> 0:18:10.720 by saying, quote, the brain is able to use incoming 0:18:10.760 --> 0:18:14.000 information from the skin as if it were coming from 0:18:14.040 --> 0:18:19.120 the eyes end quote. The subjective experience for the blind 0:18:19.240 --> 0:18:21.679 people who are feeling this in their back was that 0:18:22.200 --> 0:18:27.199 visual objects were located out there instead of on the 0:18:27.200 --> 0:18:30.359 skin of their back. In other words, it was something 0:18:30.440 --> 0:18:33.000 like vision. And think about it this way. When you're 0:18:33.040 --> 0:18:35.600 at the coffee shop and you see your friend waving 0:18:35.640 --> 0:18:38.920 at you across the way, the photons from your friend 0:18:39.119 --> 0:18:42.639 are impinging on your photoreceptors in your eye. But you 0:18:42.680 --> 0:18:46.360 don't perceive that the signal is at your eyes or 0:18:46.400 --> 0:18:48.800 in your brain. You perceive that your friend is out 0:18:48.880 --> 0:18:52.359 there waving at you from a distance. And so it 0:18:52.440 --> 0:18:56.600 goes with the users of baki Rita's modified dental chair. 0:18:57.200 --> 0:19:02.600 They were perceiving the object out there now. Amazingly. While 0:19:02.720 --> 0:19:06.160 baki Rita's device was the first to hit public consciousness, 0:19:06.480 --> 0:19:10.000 it was not actually the first attempt at sensory substitution. 0:19:10.680 --> 0:19:12.960 On the other side of the world, at the end 0:19:13.000 --> 0:19:18.280 of the eighteen nineties, a Polish ophthalmologist developed a crude 0:19:18.359 --> 0:19:22.280 device for people who were blind. He put a single 0:19:22.359 --> 0:19:25.600 photo cell on the forehead of a blind person, and 0:19:25.640 --> 0:19:29.280 the more light that hit it, the louder a sound 0:19:29.320 --> 0:19:32.880 would be in the person's ear, so based on the 0:19:33.000 --> 0:19:37.600 sound's intensity, the blind person could tell where there were 0:19:37.680 --> 0:19:40.800 lights or where there were dark areas. Unfortunately, the whole 0:19:40.800 --> 0:19:43.760 device was very large and heavy, and of course it 0:19:43.840 --> 0:19:47.200 was only one pixel of resolution, so it never got 0:19:47.200 --> 0:19:51.120 any traction. But in nineteen sixty another group in Poland 0:19:51.160 --> 0:19:54.440 picked up the ball and ran with it. They recognized 0:19:54.480 --> 0:19:58.159 that hearing is critical for the blind, so they turned 0:19:58.240 --> 0:20:03.200 to passing in the light information via touch. They built 0:20:03.240 --> 0:20:06.520 a helmet that had all these vibratory motors in it, 0:20:06.840 --> 0:20:10.959 and they essentially drew the images on the head, and 0:20:11.080 --> 0:20:15.440 blind participants were able to move around in these specially 0:20:15.480 --> 0:20:19.679 prepared rooms that were painted to enhance the contrast of 0:20:19.720 --> 0:20:24.560 the door frames and the furniture edges. It worked. Unfortunately, 0:20:25.040 --> 0:20:28.080 it was also heavy and would get very hot, and 0:20:28.160 --> 0:20:30.840 so the world had to wait. But the proof of 0:20:31.000 --> 0:20:35.280 principle was starting to emerge. Now, why did these strange 0:20:35.280 --> 0:20:39.920 approaches work. It's because input to the brain, whether that's 0:20:39.920 --> 0:20:42.320 from photons of the eyes, or air compression waves of 0:20:42.400 --> 0:20:46.040 the ears, or pressure on the skin, they're all converted 0:20:46.480 --> 0:20:50.520 into the common currency of electrical signals. So as long 0:20:50.560 --> 0:20:56.040 as the incoming spikes carry information that represents something important 0:20:56.040 --> 0:21:00.159 about the outside world, the brain will learn how to 0:21:00.240 --> 0:21:04.280 interpret it. The vast forests of your brain cells in 0:21:04.320 --> 0:21:07.760 the dark, they don't care about how the spikes get there. 0:21:08.080 --> 0:21:10.480 They just do their work on it. Now, there have 0:21:10.560 --> 0:21:15.200 been all kinds of incarnations of sensory substitution for the blind. 0:21:15.720 --> 0:21:20.120 One also from the nineteen sixties, is called the sonic glasses. 0:21:20.680 --> 0:21:22.560 It takes a video feed right in front of you 0:21:22.920 --> 0:21:26.760 and turns that into a sound landscape. So as things 0:21:27.200 --> 0:21:30.000 move around and get closer and farther, it sounds like 0:21:32.960 --> 0:21:37.879 it sounds like a cacophony. But after some time, blind 0:21:37.920 --> 0:21:42.000 people start getting really good at understanding what is in 0:21:42.040 --> 0:21:45.680 front of them just based on what they're hearing through 0:21:45.720 --> 0:21:48.919 their ears. And the best example of this is a 0:21:49.000 --> 0:21:51.359 program that you can download on your cell phone called 0:21:51.920 --> 0:21:56.080 the Voice. Note that the three middle letters are oh, 0:21:56.359 --> 0:21:59.880 I see anyway. This is developed by an engineer named 0:22:00.400 --> 0:22:04.359 Meyer in the Netherlands, and it started as a bulky project, 0:22:04.400 --> 0:22:06.560 but it can now be downloaded on your phone. You 0:22:06.600 --> 0:22:10.680 point your phone camera at things and the program converts 0:22:11.080 --> 0:22:15.960 what the phone sees into sounds. The app is amazing 0:22:16.080 --> 0:22:18.399 and you can download this onto your phone and start 0:22:18.440 --> 0:22:21.360 walking around in the world with it and really understand 0:22:21.600 --> 0:22:24.960 what's going on when you convert site into sound. And 0:22:25.000 --> 0:22:28.040 my colleagues all over the world, like Jamie Warren and 0:22:28.080 --> 0:22:32.200 emirah Medi, have been running science experiments on these sorts 0:22:32.240 --> 0:22:35.800 of approaches. And by the way, the century substitution doesn't 0:22:35.840 --> 0:22:39.280 have to be through the ears. Another version is called 0:22:39.760 --> 0:22:43.240 the brain port, and this is a little grid. It's 0:22:43.280 --> 0:22:46.960 called an electro techtile grid. It sits on your tongue 0:22:47.280 --> 0:22:50.600 and gives little shocks. So you have a camera and 0:22:50.680 --> 0:22:54.080 that video feed gets turned into these little shocks on 0:22:54.119 --> 0:22:56.880 your tongue. It feels like pop rocks in your mouth. 0:22:57.200 --> 0:22:59.920 And blind people can get so good at using this 0:23:00.080 --> 0:23:03.240 that they can throw a ball into a basket where 0:23:03.280 --> 0:23:07.040 they can navigate a complex obstacle. Course they can come 0:23:07.080 --> 0:23:12.760 to see through their tongue. Now that sounds completely insane, right, 0:23:13.040 --> 0:23:18.119 but remember all vision, ever is, are these electrical signals 0:23:18.160 --> 0:23:21.560 coursing around in your brain. Your brain doesn't know where 0:23:21.560 --> 0:23:25.959 the signals come from, it just figures out what to 0:23:26.040 --> 0:23:29.280 do with them. So my laboratory set out some years 0:23:29.320 --> 0:23:34.320 ago to solve sensory substitution for people who are deaf, 0:23:34.960 --> 0:23:37.120 and we wanted to make it so that the sound 0:23:37.200 --> 0:23:39.600 from the world gets converted in some ways so that 0:23:39.680 --> 0:23:43.480 a deaf person can understand what is being said. So 0:23:43.560 --> 0:23:48.000 with my graduate student, Scott Novic, we built a vest. 0:23:48.240 --> 0:23:50.320 Now this is not a normal vest. This is a 0:23:50.440 --> 0:23:53.760 vest that zips up tight around the torso and it 0:23:53.800 --> 0:23:58.280 has thirty two little motors on it. And these are 0:23:58.520 --> 0:24:01.480 vibratory motors like the buzzer on your cell phone, but 0:24:01.680 --> 0:24:05.520 thirty two of them, and they're distributed pretty evenly around 0:24:05.560 --> 0:24:09.000 your waist in your back, and each motor represents a 0:24:09.080 --> 0:24:14.359 different frequency of sound from low to high. And by 0:24:14.440 --> 0:24:17.440 breaking up sound in this way, this is the same 0:24:17.480 --> 0:24:20.280 thing that your inner ear does a part of your 0:24:20.280 --> 0:24:25.119 interear called the cochlea, So we have essentially transferred the 0:24:25.200 --> 0:24:29.720 cochlea to the torso, so it captures sound and turns 0:24:29.720 --> 0:24:33.000 that into these patterns vibration. So some years ago we 0:24:33.040 --> 0:24:36.280 started to test this in conjunction with the deaf community. 0:24:36.720 --> 0:24:39.760 Our first participant was a guy named Jonathan. He was 0:24:39.800 --> 0:24:42.680 thirty seven years old. He had a master's degree, and 0:24:42.720 --> 0:24:46.639 he had been born profoundly deaf, which means there was 0:24:46.680 --> 0:24:49.880 a part of his umvelt that was unavailable to him. 0:24:50.359 --> 0:24:53.479 So we had Jonathan wear the vest and train with 0:24:53.520 --> 0:24:56.640 it for four days, two hours a day, and by 0:24:56.680 --> 0:25:00.280 the fifth day he was pretty good at identifying the 0:25:00.320 --> 0:25:03.120 words that were being said to him. So you say 0:25:03.160 --> 0:25:06.960 the word dog, and Jonathan feels a pattern of vibrations 0:25:06.960 --> 0:25:09.480 all over the vest, and his job is simply to 0:25:09.640 --> 0:25:12.879 write on the dry erase board what he thinks the 0:25:12.920 --> 0:25:15.359 word might have been, and by day five, he could 0:25:15.359 --> 0:25:18.280 get this mostly right. Now. We had trained him on 0:25:18.320 --> 0:25:21.399 a limited number of words, what's called a closed set, 0:25:21.880 --> 0:25:24.119 but when we switched to a new set of words, 0:25:24.160 --> 0:25:26.720 once he had never heard before, he was able to 0:25:26.800 --> 0:25:29.760 perform well above chance. And he learned more and more 0:25:29.840 --> 0:25:32.840 quickly with every new set. And this suggested he wasn't 0:25:32.920 --> 0:25:38.199 just memorizing some answers. He was actually learning how to 0:25:38.560 --> 0:25:42.959 hear with the vest. He was translating the complicated pattern 0:25:43.000 --> 0:25:49.040 of vibrations into an understanding of what was being said. Now, 0:25:49.080 --> 0:25:52.800 he wasn't doing this consciously, because the patterns are too 0:25:52.960 --> 0:25:57.080 complicated for that, but his brain was unlocking the meaning 0:25:57.119 --> 0:26:00.000 of this. And by the way, this is just like you. 0:26:00.000 --> 0:26:04.240 So listening to this podcast, you're not thinking, oh, Eagelman 0:26:04.359 --> 0:26:06.560 is saying some high frequencies and now some low and 0:26:06.640 --> 0:26:10.760 some medium, so that must be a s sound. Instead, 0:26:10.880 --> 0:26:14.560