WEBVTT - Rerun: Augmenting Your Reality 0:00:04.240 --> 0:00:07.240 Welcome to tech Stuff, a production of I Heart Radios 0:00:07.320 --> 0:00:14.120 How Stuff Works. Hey there, and welcome to tech Stuff. 0:00:14.200 --> 0:00:16.880 I'm your host, Jonathan Strickland. I'm an executive producer with 0:00:16.960 --> 0:00:19.800 iHeart Radio and I love all things tech, and we're 0:00:19.800 --> 0:00:23.239 going to enjoy another classic episode of tech stuff in 0:00:23.239 --> 0:00:26.920 this episode. And uh, I promised new ones are right 0:00:26.920 --> 0:00:29.360 around the corner. I've got a cool one coming up 0:00:29.400 --> 0:00:32.159 in just a little while that you definitely want to 0:00:32.240 --> 0:00:37.160 check out. Very important, all about the dangers of browsing 0:00:37.159 --> 0:00:40.519 the Internet in an insecure way. But before we get 0:00:40.520 --> 0:00:43.640 to that, I thought maybe we could revisit this classic 0:00:43.680 --> 0:00:48.160 episode in which I talk about augmented reality. So this 0:00:48.240 --> 0:00:52.080 episode is titled Augmenting Your Reality, and it originally aired 0:00:52.120 --> 0:00:54.400 just a couple of years ago, and I thought it'd 0:00:54.440 --> 0:00:56.800 be fun to revisit it. So let's sit back and 0:00:56.920 --> 0:01:01.480 enjoy this classic episode. So I thought I would do 0:01:01.800 --> 0:01:06.920 a deeper dive, a bigger explanation about what augmented reality is, 0:01:06.959 --> 0:01:09.720 what it's all about, how it works, and sort of 0:01:09.760 --> 0:01:15.560 the applications we might put a r toward things that 0:01:15.720 --> 0:01:18.440 you know, was it good for tons of stuff? As 0:01:18.440 --> 0:01:22.000 it turns out, So the first thing we should do 0:01:22.600 --> 0:01:26.520 is probably defined some terms, because if you haven't really 0:01:26.520 --> 0:01:30.080 looked into augmented reality and you aren't familiar with a R, 0:01:30.520 --> 0:01:33.680 you might just be lost. I'm gonna define it all 0:01:33.720 --> 0:01:36.320 for you right now, because that's the kind of stand 0:01:36.400 --> 0:01:41.000 up guy I am. Technically speaking, augmented reality is using 0:01:41.040 --> 0:01:45.840 digital information to enhance or augment and experience in our 0:01:45.880 --> 0:01:51.000 physical real world. So the way we usually see this 0:01:51.160 --> 0:01:56.120 implemented involves some sort of display that has an image 0:01:56.120 --> 0:01:58.800 of the real world on it and it overlays digital 0:01:58.840 --> 0:02:01.360 information on top of that itch. So think of like 0:02:01.880 --> 0:02:06.160 a camera's viewfinder, like an LCD screen on a camera, 0:02:06.280 --> 0:02:09.480 and it actually labels the buildings that are in view. 0:02:09.520 --> 0:02:11.400 When you're out on the street and you hold the 0:02:11.440 --> 0:02:15.760 camera up or a smartphone or even a wearable device 0:02:15.840 --> 0:02:18.840 like a head mounted display that you can look through 0:02:19.040 --> 0:02:21.120 so you can see the real world, you're not just 0:02:21.160 --> 0:02:23.840 staring at a screen, or if you are staring at 0:02:23.880 --> 0:02:26.720 a screen, you're staring at a video feed that is 0:02:26.760 --> 0:02:29.560 provided by an external camera amounted just on the other 0:02:29.600 --> 0:02:31.800 side of the screen. So it's like you're looking through 0:02:32.120 --> 0:02:34.840 a display in the first place, but then on top 0:02:34.960 --> 0:02:38.760 of that view you have this digital information. That's the 0:02:38.800 --> 0:02:41.600 most common implementation we talked about, but it's not the 0:02:41.639 --> 0:02:44.560 only one. Augmented reality does not have to only be 0:02:44.880 --> 0:02:49.600 or even involve visual information at all. You could have 0:02:49.720 --> 0:02:54.000 audio only augmented reality, for example. But the whole idea 0:02:54.160 --> 0:02:58.519 is that it's something that's created digitally to enhance your 0:02:58.720 --> 0:03:01.799 experience in the real world. Now we can contrast this 0:03:02.320 --> 0:03:07.079 with the concept of virtual reality. Virtual reality, of course, 0:03:07.680 --> 0:03:11.480 is a term where you create an experience completely through 0:03:11.800 --> 0:03:16.240 computer generated means computer is making all the things you 0:03:16.280 --> 0:03:20.280 see and here, and maybe even beyond that if you 0:03:20.360 --> 0:03:23.960 have really sophisticated UH setups, so you might have some 0:03:24.040 --> 0:03:27.400 haptic feedback. Haptic refers to your sense of touch, So 0:03:27.440 --> 0:03:30.480 if you have haptic feedback, that means you're getting information 0:03:30.520 --> 0:03:34.000 feedback through your sense of touch. Common example of this 0:03:34.520 --> 0:03:38.280 is a rumble pack inside a game controller, where you 0:03:38.320 --> 0:03:40.440 know you fire a gun and a first person shooter 0:03:40.880 --> 0:03:44.800 and your controller rumbles as a result, letting you know 0:03:44.920 --> 0:03:50.640 that you are in fact, unleashing virtual destruction upon all 0:03:50.720 --> 0:03:54.240 you survey. Well, the same thing can be true with 0:03:54.800 --> 0:03:58.680 virtual reality set up. So virtual reality is all about 0:03:58.720 --> 0:04:04.480 constructing an artificial reality a simulated reality. Augmented reality is 0:04:04.520 --> 0:04:07.680 all about enhancing the one that we are actually in. 0:04:08.200 --> 0:04:13.160 And then there's also mixed reality. Mixed reality is kind 0:04:13.160 --> 0:04:16.040 of sort of in between the two. You might have 0:04:16.160 --> 0:04:21.240 some physical objects within a room that are also mapped 0:04:21.279 --> 0:04:25.279 to a virtual environment, and then you use something like 0:04:25.320 --> 0:04:28.880 a head molded display to enter the virtual environment. That's 0:04:28.880 --> 0:04:32.159 what looks like you're inside, but you have physical objects 0:04:32.160 --> 0:04:34.520 in the room around you that are also mapped to 0:04:34.560 --> 0:04:37.040 the virtual world, Meaning you could pick up this physical 0:04:37.080 --> 0:04:40.560 object and you would see that reflected within the virtual world, 0:04:40.600 --> 0:04:43.480 where you might pick up a sword and shield or 0:04:43.800 --> 0:04:48.320 move a chair or something along those lines. So augmented reality, 0:04:48.400 --> 0:04:52.360 virtual reality, and mixed reality are all kind of inter related, 0:04:52.680 --> 0:04:56.360 so much so that their histories also are very much 0:04:56.440 --> 0:05:00.360 inter related. And there's some people who try to elect 0:05:00.480 --> 0:05:05.080 these different technologies, these different approaches and put them under 0:05:05.120 --> 0:05:08.120 a common umbrella, and they tend to use the phrase 0:05:08.520 --> 0:05:12.880 alternate reality, which is unfortunate because that's also a r 0:05:13.000 --> 0:05:15.960 but alternate reality is kind of the umbrella for virtual, 0:05:16.240 --> 0:05:20.360 augmented and mixed reality. Uh that that that kind of 0:05:20.400 --> 0:05:23.760 gives you the definition of those basic terms, and it 0:05:23.839 --> 0:05:26.719 is important to understand them because they're becoming more and 0:05:26.760 --> 0:05:30.839 more important today. You are already probably aware of a 0:05:30.880 --> 0:05:33.760 lot of VR headsets that are out there on the 0:05:33.800 --> 0:05:38.320 market as well as VR uh Well, they're they're kind 0:05:38.320 --> 0:05:41.520 of like cases that you slide your smartphone into, so 0:05:41.560 --> 0:05:46.599 your smartphone becomes the actual display on a VR headset. 0:05:46.600 --> 0:05:50.000 The headset itself is more or less just a head 0:05:50.080 --> 0:05:53.599 mounted case for your phone. We've seen a lot of 0:05:53.600 --> 0:05:56.200 those come out over the last few years. We've also 0:05:56.240 --> 0:05:59.359 seen a lot of a R applications come out, typically 0:05:59.480 --> 0:06:03.039 for things iPads and smartphones, but we've also seen some 0:06:03.080 --> 0:06:08.240 hardware come out that for wearable devices that falls into 0:06:08.279 --> 0:06:11.040 the augmented reality category, stuff like Google Glass, which i'll 0:06:11.040 --> 0:06:13.560 talk about more a little bit later in this episode. 0:06:14.920 --> 0:06:19.640 For augmented reality to work, to get this enhanced experience 0:06:19.760 --> 0:06:23.440 of reality around you, there are a lot of technological 0:06:23.480 --> 0:06:27.880 components that have to come together so that you actually 0:06:27.920 --> 0:06:31.960 do get an experience that is meaningful. You know, you 0:06:31.960 --> 0:06:35.279 you have to have technology that quote unquote knows where 0:06:35.320 --> 0:06:37.840 you are and what you are looking at or what 0:06:37.920 --> 0:06:41.400 you are close to in order to get that augmented experience. 0:06:41.760 --> 0:06:44.120 It wouldn't do me any good. If I put on 0:06:44.440 --> 0:06:49.120 an augmented reality headset, for example, and stared at let's 0:06:49.160 --> 0:06:53.680 say a famous painting, and instead of getting information about 0:06:53.680 --> 0:06:56.600 the famous painting, I see an exploded view of an 0:06:56.760 --> 0:07:00.080 car engine, that would make no sense. So you have 0:07:00.120 --> 0:07:03.360 to build in technologies in order for the a R 0:07:03.480 --> 0:07:07.120 to understand what it is you're trying to do and 0:07:07.240 --> 0:07:10.560 to augment that experience, which meant that we had to 0:07:10.600 --> 0:07:15.480 wait a pretty good long time for the various technologies 0:07:16.080 --> 0:07:21.000 that we used to to create this relationship to mature 0:07:21.200 --> 0:07:24.640 to a point where it was possible. Sometimes we had 0:07:24.680 --> 0:07:26.760 technologies that would allow us to do it, but it 0:07:26.920 --> 0:07:32.240 required uh tethering headsets to very large computers, which meant 0:07:32.240 --> 0:07:35.600 that you didn't have really any mobility, and uh it 0:07:35.720 --> 0:07:41.840 really limited the usefulness of the actual application. In other cases, 0:07:42.600 --> 0:07:45.680 you could say things like your head tracking technology was 0:07:45.720 --> 0:07:49.040 absolutely necessary for a R to develop the way it did. 0:07:49.880 --> 0:07:54.000 GPS technology as well. Remember it wasn't that long ago 0:07:54.520 --> 0:07:59.760 that we ordinary mere mortals didn't have access to really 0:08:00.000 --> 0:08:04.880 accurate GPS information. For a very long time, that was 0:08:04.960 --> 0:08:10.000 purposefully made less accurate. It was a matter of national defense. 0:08:10.680 --> 0:08:13.400 It wasn't until the nineties that you started to see 0:08:13.400 --> 0:08:19.600 GPS become more accurate for the basic consumer. Wait, back 0:08:19.640 --> 0:08:22.840 in the day, you might get accuracy of up to 0:08:22.920 --> 0:08:25.440 around a hundred meters, which is not great if you're 0:08:25.480 --> 0:08:28.120 looking for the next place to make your turn. If 0:08:28.200 --> 0:08:31.800 it's a hundred meters away, that's that's pretty far. But 0:08:31.920 --> 0:08:34.160 now it's within a few feet, so it's much better. 0:08:34.679 --> 0:08:36.880 That sort of stuff all had to come together in 0:08:37.000 --> 0:08:42.640 order for augmented reality to become a viable I almost 0:08:42.640 --> 0:08:45.800 said a reality, but that just starts to sound redundant 0:08:46.720 --> 0:08:49.960 at any rate. Let's talk about some of these technologies 0:08:50.240 --> 0:08:55.080 we are We really need things like gyroscopes, accelerometers. These 0:08:55.120 --> 0:09:00.680 help devices understand their orientation where they are in respect 0:09:00.760 --> 0:09:04.079 to something else, like are they for a smartphone it 0:09:04.160 --> 0:09:06.280 might be is it in landscape mode or portrait mode? 0:09:06.720 --> 0:09:10.679 But for a head mounted display, it would help give 0:09:10.840 --> 0:09:13.680 the the unit the information that needs to know which 0:09:13.679 --> 0:09:15.920 way you're looking, like are you looking to the east 0:09:16.000 --> 0:09:19.520 or to the west. That kind of thing. UH also 0:09:19.640 --> 0:09:24.760 compasses obviously very important GPS sensors, image recognition software, but 0:09:24.800 --> 0:09:27.720 has become really important so that when you are looking 0:09:27.880 --> 0:09:32.400 at something UH, the system can actually identify what that is. 0:09:32.640 --> 0:09:35.319 In some cases you can get around this. You can 0:09:35.360 --> 0:09:38.280 design an a R system where let's say you make 0:09:38.320 --> 0:09:43.680 a movie poster and the a R application has the 0:09:43.679 --> 0:09:46.840 movie poster or animate in some way if you hold 0:09:46.960 --> 0:09:50.960 up a smartphone that's running the appropriate app. So I'm 0:09:51.000 --> 0:09:54.360 just gonna take a movie from my past that does 0:09:54.400 --> 0:09:57.360 not have an a R movie poster or associated with it, 0:09:57.960 --> 0:09:59.880 but one that I can talk about as if it. 0:10:00.000 --> 0:10:02.400 We're a good example, and that has to be Big 0:10:02.440 --> 0:10:06.600 Trouble in Little China, universally declared the best movie that 0:10:06.640 --> 0:10:09.240 has ever been made. So you've got your Big Trouble 0:10:09.320 --> 0:10:12.000 and Little China poster up on the wall, and you 0:10:12.040 --> 0:10:15.320 hold up your smartphone and you activate your Big Trouble 0:10:15.360 --> 0:10:20.040 and Little China movie marketing app, and the camera on 0:10:20.080 --> 0:10:22.760 your phone detects the poster it you know the posters 0:10:22.800 --> 0:10:26.679 there well. The app and the poster together are able 0:10:26.720 --> 0:10:30.600 to construct the augmented experience because there have been elements 0:10:30.600 --> 0:10:33.360 put into the poster that the app is looking for. 0:10:34.200 --> 0:10:37.600 And once the app identifies that, like, it sees maybe 0:10:37.840 --> 0:10:40.560 eight different points on the poster, and because of the 0:10:40.600 --> 0:10:44.200 orientation of those points, it knows what angle it's at 0:10:44.280 --> 0:10:47.319 what height it's at in relation to the phone, and 0:10:47.360 --> 0:10:52.199 can give you on your display the augmented reality experience. 0:10:52.240 --> 0:10:55.480 In this case, it's obviously Jack Burton and the pork 0:10:55.559 --> 0:10:59.640 Chop Express eating a sandwich, because, as we know, the 0:10:59.679 --> 0:11:04.000 most riveting scene in the movie unfolds in this way. 0:11:04.080 --> 0:11:07.440 So that would be kind of an augmented reality experience 0:11:07.440 --> 0:11:11.240 where you didn't have to worry about every possible application 0:11:11.280 --> 0:11:13.560 down in the real world. You made it for something 0:11:13.679 --> 0:11:16.760 very specific, which means in your software you can have 0:11:16.840 --> 0:11:22.320 the camera look quote unquote for these particular points of 0:11:22.360 --> 0:11:26.680 reference and thus create the augmented experience in that way. 0:11:26.880 --> 0:11:28.800 If you want to take that and move it to 0:11:28.880 --> 0:11:32.760 the real world where you can see augmented information about 0:11:32.920 --> 0:11:35.960 just the world around you, it becomes way more complicated. 0:11:36.320 --> 0:11:39.800 You have to have very sophisticated image recognition software so 0:11:39.840 --> 0:11:44.199 that the camera picks up the images, the software processes 0:11:44.240 --> 0:11:47.360 the information, identifies what those images are, and gives you 0:11:47.360 --> 0:11:52.240 the relevant information. So working with all the sensors, augmented 0:11:52.240 --> 0:11:55.319 reality can make this a possibility. So another example, let's 0:11:55.320 --> 0:11:58.559 say you're out on the street in Atlanta. You're here 0:11:58.600 --> 0:12:01.680 in my hometown at and a Georgia and you're looking 0:12:01.720 --> 0:12:03.400 at a building and you wonder what it is, and 0:12:03.440 --> 0:12:05.440 you hold up your phone and you've got your little 0:12:05.800 --> 0:12:08.840 map app that allows you to look at a real 0:12:08.920 --> 0:12:12.240 world setting and tells you information about it, and it 0:12:12.280 --> 0:12:14.760 tells you it's the Georgia Aquarium. Well, first of all, 0:12:14.760 --> 0:12:16.720 you would probably know that already because the signage there 0:12:16.800 --> 0:12:20.400 is actually pretty good. But the point being that this 0:12:20.440 --> 0:12:24.040 would be something that would tap into the GPS cordints 0:12:24.080 --> 0:12:26.280 on your phone, so it would know where your location 0:12:26.440 --> 0:12:29.040 was and help narrow that down. The compass would tell 0:12:29.040 --> 0:12:33.840 it what direction you are facing the the camera angle. 0:12:33.880 --> 0:12:37.640 Also we have some image recognition going on there. The 0:12:37.640 --> 0:12:41.520 accelerometer tells the orientation of the phone itself. All of 0:12:41.520 --> 0:12:44.920 this data together would give the software the information needed 0:12:45.280 --> 0:12:50.640 for it to display the label Georgia Aquarium on your phone. 0:12:50.640 --> 0:12:55.120 And it all happens in an instant that's pretty amazing. Typically, 0:12:55.160 --> 0:12:59.200 you also have to have some other method to communicate 0:12:59.320 --> 0:13:03.280 with a larger infrastructure, because we don't have the capability 0:13:03.400 --> 0:13:07.560 of building an enormously powerful computer that has all this 0:13:07.720 --> 0:13:11.040 real world information programmed into it and make it a 0:13:11.200 --> 0:13:15.440 handheld or wearable device. So usually you have to pair 0:13:15.640 --> 0:13:19.400 these devices with some other larger infrastructure. Sometimes it's a 0:13:19.440 --> 0:13:23.360 double handshake. For example, with Google Glass, you would use 0:13:23.360 --> 0:13:26.800 Bluetooth to connect Google Glass to a smartphone. Then the 0:13:26.840 --> 0:13:30.680 smartphone would have the connection to the larger internet through 0:13:30.760 --> 0:13:38.760 your smartphones UH cell service provider. So while you're experiencing 0:13:39.600 --> 0:13:43.080 the augmented reality through the Google Glass, it's actually communicating 0:13:43.080 --> 0:13:45.559 through your phone to the infrastructure to get the data 0:13:45.600 --> 0:13:48.040 it needs to show you the information. It's showing you 0:13:49.000 --> 0:13:51.720 very important elements. And all of these components, like I said, 0:13:51.840 --> 0:13:55.600 came together more or less around the same time. Most 0:13:55.640 --> 0:13:59.040 of them were being developed independently of each other, and 0:13:59.080 --> 0:14:01.960 it's just that now we're seeing them all converge. That's 0:14:01.960 --> 0:14:05.840 an old favorite word here at tech stuff converge together 0:14:05.880 --> 0:14:08.840 to create the augmented reality experience and make it possible. 0:14:09.640 --> 0:14:11.800 So how did we get here? How did these different 0:14:11.800 --> 0:14:16.000 elements develop? Well, there are a whole bunch of technology 0:14:16.000 --> 0:14:19.440 pioneers who really create the foundation for augmented reality as 0:14:19.440 --> 0:14:22.440 well as virtual reality and mixed reality. But one that 0:14:22.520 --> 0:14:25.040 I think we really need to concentrate on at first 0:14:25.120 --> 0:14:29.960 is Ivan Sutherland. Now. Sutherland was born in Hastings, Nebraska 0:14:30.040 --> 0:14:33.080 in nineteen thirty eight, and as a kid, he was 0:14:33.160 --> 0:14:39.400 fascinated with mathematics, particularly geometry, and also with engineering. He 0:14:39.440 --> 0:14:42.800 began to study and experiment with computers while he was 0:14:42.840 --> 0:14:45.440 in school, and this was at a time where personal 0:14:45.440 --> 0:14:48.160 computers weren't a thing. There were no personal computers at 0:14:48.160 --> 0:14:50.720 this point. Computers were actually pretty rare, and they were huge, 0:14:51.280 --> 0:14:55.320 and in fact, they often would rely upon physical media 0:14:55.400 --> 0:14:59.320 formats like punch punch cards or paper tape to read 0:14:59.400 --> 0:15:03.320 a program him. So you didn't even have a disc 0:15:03.480 --> 0:15:07.160 or like certainly nothing like a USB thumb drive or 0:15:07.200 --> 0:15:09.920 anything like that. You you actually had to put physical 0:15:10.000 --> 0:15:13.240 media into the machine for it to read and then 0:15:13.360 --> 0:15:18.040 execute whatever program you had designed for that device. He 0:15:18.280 --> 0:15:21.720 went to college at what is now Carnegie Mellon University 0:15:21.760 --> 0:15:24.440 on a full scholarship. He graduated with a Bachelor of 0:15:24.480 --> 0:15:26.440 Science degree. He would then go on to earn a 0:15:26.480 --> 0:15:29.800 master's degree at cal Tech and a PhD in electrical 0:15:29.880 --> 0:15:34.000 engineering from m I T And actually his doctoral thesis 0:15:34.000 --> 0:15:38.560 supervisor was Claude Shannon. And we talked about Claude Shannon 0:15:38.600 --> 0:15:40.720 back in the two thousand and fourteen episode Who Is 0:15:40.760 --> 0:15:44.960 Claude Shannon. Um. We recorded that not too long after 0:15:45.120 --> 0:15:48.280 Shannon's passing, So if you want to hear a really 0:15:48.320 --> 0:15:52.280 interesting story about a pioneer in computer science, you should 0:15:52.280 --> 0:15:55.360 go check out that two thousand and fourteen episode. Back 0:15:55.400 --> 0:15:59.720 to Sutherland. For his thesis, he created something called sketch 0:15:59.760 --> 0:16:02.720 pack AD and that was really, by most accounts, the 0:16:02.800 --> 0:16:07.520 first computer graphical user interface, or gooey. A graphical user 0:16:07.560 --> 0:16:13.520 interface means that you interact with the computer through graphics 0:16:13.600 --> 0:16:17.560 representing various commands on the computer. Windows and the Mac 0:16:17.560 --> 0:16:21.720 operating system are both examples of graphical user interfaces, as 0:16:21.760 --> 0:16:23.880 is the interface on your smartphone. If you have a 0:16:23.920 --> 0:16:28.760 smartphone where you choose applications on a screen, that's a 0:16:28.800 --> 0:16:32.520 graphical user interface. Well, Sutherland created what is largely considered 0:16:32.560 --> 0:16:35.320 to be the first one of those. After college, he 0:16:35.400 --> 0:16:38.720 entered military service and he was assigned to the National 0:16:38.760 --> 0:16:43.880 Security Agency. We have great friends there. I assume I'm 0:16:43.880 --> 0:16:48.080 sure they're listening, because they're listening to everything at any rate. 0:16:48.160 --> 0:16:50.680 He entered the n s A as an electrical engineer, 0:16:50.800 --> 0:16:53.880 and in nineteen sixty four he replaced J. C. R. 0:16:54.000 --> 0:16:58.040 Lick Lighter as the head of DARPA's Information Processing Techniques 0:16:58.160 --> 0:17:03.240 Office or ip TO. And also by back then DARPA 0:17:03.360 --> 0:17:07.600 wasn't DARPA, it was just ARPA. Uh so this is 0:17:07.640 --> 0:17:09.439 the same group, by the way, that would end up 0:17:09.480 --> 0:17:12.720 doing a lot of work that would form the ARPANETTE 0:17:12.880 --> 0:17:15.800 a few years later, and the Arpanette was the predecessor 0:17:15.920 --> 0:17:19.560 to the Internet in some ways. At least, the ARPANET 0:17:19.720 --> 0:17:23.760 was what ended up being the building blocks for the 0:17:23.800 --> 0:17:27.439 infrastructure that would become the Internet. Now, all of that 0:17:27.480 --> 0:17:32.159 work happened after Sutherland had already departed the organization. His 0:17:32.200 --> 0:17:37.280 work became a fundamental component of both virtual and augmented reality. 0:17:37.320 --> 0:17:40.919 As I mentioned earlier in n he wrote a piece 0:17:41.000 --> 0:17:43.280 and essay. It's very short, it's very easy read, and 0:17:43.320 --> 0:17:45.680 you can find it online. The title of the essay 0:17:45.800 --> 0:17:48.960 is the Ultimate Display. And if you ever do any 0:17:49.040 --> 0:17:53.320 research and virtual reality or augmented reality, this essay is 0:17:53.359 --> 0:17:56.280 going to pop up in your research, so go ahead 0:17:56.280 --> 0:17:58.199 and read it. It's like two pages long, so it 0:17:58.240 --> 0:18:03.040 goes very quickly. In that essay he talked about several ideas, 0:18:03.119 --> 0:18:10.000 including the idealized display, the ultimate display, something that would 0:18:10.040 --> 0:18:15.200 be the furthest you could go with display technology. Now, 0:18:15.280 --> 0:18:18.440 keep in mind in his time, by his time, he's 0:18:18.440 --> 0:18:20.439 still alive, by the way, but this time in the 0:18:20.480 --> 0:18:25.440 nineties sixties, he uh, you know, things were just restricted 0:18:25.480 --> 0:18:30.159 to monitors. You might have a light pen, but usually 0:18:30.200 --> 0:18:33.760 you would just use a keyboard. Like it was pretty 0:18:33.760 --> 0:18:36.880 bare bones. But he said, let's push this as far 0:18:36.920 --> 0:18:40.760 as we can imagine it. And in his example, he 0:18:40.840 --> 0:18:44.840 thought of a room that would be completely controlled by computers. 0:18:44.880 --> 0:18:48.680 Everything you would experience within that room would be generated 0:18:48.720 --> 0:18:53.800 by a computer. Everything you see here, smell, taste, and touch, 0:18:54.680 --> 0:18:57.240 all of it generated by computers. The computer would even 0:18:57.280 --> 0:19:01.520 be able to form physical objects out of ror matter itself. Now, 0:19:01.560 --> 0:19:05.200 he wasn't suggesting that this would ever be a device 0:19:05.280 --> 0:19:07.280 that we would actually be able to build. He was 0:19:07.320 --> 0:19:13.000 just saying, what is the ultimate incarnation of display technology? 0:19:13.280 --> 0:19:16.080 And if you read it, you realize, oh, this is 0:19:16.080 --> 0:19:18.760 where the Star Trek Next Generation writers got their idea 0:19:18.800 --> 0:19:22.359 for the Holidack. But unlike Star Trek the Next Generation, 0:19:22.520 --> 0:19:25.760 the Ultimate Display would not go on the fritz every 0:19:25.760 --> 0:19:28.760 other episode and try to kill the crew. It was 0:19:28.800 --> 0:19:34.480 better than that. The Ultimate Display was sort of a 0:19:33.960 --> 0:19:39.480 a foundational like Philosophically, it was foundational for virtual reality 0:19:39.520 --> 0:19:43.320 and augmented reality. This idea of a very immersive experience 0:19:43.400 --> 0:19:48.280 where you, as a user are surrounded somehow by this 0:19:49.280 --> 0:19:53.600 computer generated experience. And that's true both with augmented reality 0:19:53.640 --> 0:19:58.119 and virtual reality and augmented reality. The real world is 0:19:58.160 --> 0:20:01.840 still there, but you get this enhanced experience that is 0:20:01.880 --> 0:20:09.000 completely computer generated. So in nineteen sixty eight, Sutherland and 0:20:09.240 --> 0:20:13.359 a student named Danny Cohen would create a VR, a 0:20:13.720 --> 0:20:18.680 R head mounted display or h MD, and they nicknamed 0:20:18.720 --> 0:20:23.000 it the Sword of Damocles. Why because you had to 0:20:23.040 --> 0:20:25.280 suspend it from the ceiling. It was too heavy to 0:20:25.359 --> 0:20:28.240 wear on your head. You needed it to be nice 0:20:28.280 --> 0:20:33.000 and sturdy. Uh. It included transparent lenses, which meant you 0:20:33.040 --> 0:20:36.920 could overlay computer information on the lenses themselves, and thus 0:20:36.960 --> 0:20:39.520 you could look through the lenses at the real world 0:20:39.880 --> 0:20:43.880 and have these wire framed graphics on top of what 0:20:43.920 --> 0:20:46.639 you were looking at. And it also had a magnetic 0:20:46.680 --> 0:20:50.479 tracking system, meaning that it had sensors that could detect 0:20:50.520 --> 0:20:53.520 magnetic fields, and as you turned your head or you 0:20:54.040 --> 0:20:57.399 change the inclination of your head, it would change the 0:20:57.440 --> 0:21:01.040 magnetic field and this would be relayed as a command 0:21:01.280 --> 0:21:05.880 to the visual center. The actual lenses themselves, so that 0:21:06.000 --> 0:21:09.400 it would the change would be reflected in what you saw. 0:21:10.240 --> 0:21:12.480 So if you have a virtual environment and you turn 0:21:12.520 --> 0:21:14.920 your head to the left, you want the view within 0:21:14.960 --> 0:21:17.480 the virtual environment to go to the left too, But 0:21:17.560 --> 0:21:20.480 without head tracking technology that's impossible. So this was a 0:21:20.600 --> 0:21:23.399 very early example of head tracking technology. And again it 0:21:23.520 --> 0:21:27.560 used magnets magnetic fields in order to do that. Uh. 0:21:28.160 --> 0:21:32.480 Obviously it's also really important for augmented reality. Again, if 0:21:32.520 --> 0:21:35.600 the A R system doesn't detect that you are looking around, 0:21:36.160 --> 0:21:39.840 then you're not getting relevant information, not for the specific 0:21:39.880 --> 0:21:43.840 thing you are looking at. Anyway, as I said, the 0:21:43.840 --> 0:21:46.520 graphics were pretty primitive. There were wire frame drawings, but 0:21:46.600 --> 0:21:52.399 they still showed that this was a viable approach to 0:21:52.480 --> 0:21:57.200 technology using HMD for augmented or virtual reality use. Oh 0:21:57.240 --> 0:21:59.560 and one other note I should make so a lot 0:21:59.600 --> 0:22:01.960 of people say the sort of Damocles was the first 0:22:02.000 --> 0:22:04.800 head mounted display, and they say, you know, this is 0:22:04.840 --> 0:22:09.000 the first HMD was made in nineteen I take issue 0:22:09.080 --> 0:22:12.920 with that. I don't think of the sort of Damocles 0:22:13.080 --> 0:22:16.359 as the first head mounted display. That to me should 0:22:16.359 --> 0:22:20.760 go to a different invention called the head site h 0:22:20.800 --> 0:22:23.679 E A D s I g h T. Now that 0:22:23.800 --> 0:22:27.720 was developed by Phil Coo, and unlike the sort of Damocles, 0:22:28.280 --> 0:22:32.679 it didn't create a virtual world. Instead, the head site 0:22:32.920 --> 0:22:37.400 was sort of a remote viewfinder for a video camera. 0:22:38.440 --> 0:22:43.120 So imagine that you've got a camera mounted on a mechanical, 0:22:43.800 --> 0:22:48.000 uh swiveling mount, so you can move it left right, 0:22:48.040 --> 0:22:51.679 you can change the orientation the inclination as well, and 0:22:51.720 --> 0:22:54.520 then you have that mapped to a head mounted display, 0:22:54.640 --> 0:22:56.879 so that if I put the display on and I 0:22:56.920 --> 0:22:59.440 looked to the left, the camera pans to the left. 0:22:59.600 --> 0:23:01.360 If I to the right, it pans to the right. 0:23:01.440 --> 0:23:04.879 That sort of thing. It was meant to be a 0:23:04.920 --> 0:23:08.359 way for for people to operate a camera in a 0:23:08.400 --> 0:23:12.280 remote location that might not be very friendly to a 0:23:12.400 --> 0:23:16.520 human being standing there. For example, the exterior of an aircraft. 0:23:16.880 --> 0:23:18.960 You could have a camera mounted on the outside of 0:23:18.960 --> 0:23:22.520 your aircraft that would allow an engineer on the inside 0:23:22.960 --> 0:23:27.439 to look around and maybe help a pilot land or 0:23:27.680 --> 0:23:31.280 navigate in a dangerous situation, or just get an idea 0:23:31.440 --> 0:23:35.880 of the status of the aircraft itself. This was very 0:23:35.960 --> 0:23:40.440 much a technology that was being pushed by the military, 0:23:41.440 --> 0:23:45.320 an idea to create more military uses using this technology. 0:23:45.359 --> 0:23:50.040 To make the military more competent, more adept at very 0:23:50.119 --> 0:23:55.959 rapid changing situations on the technology front. So headsite proceeded 0:23:56.400 --> 0:23:59.240 the sort of Damocles by about seven years. That came 0:23:59.240 --> 0:24:01.879 out around night sixty one. But again it wasn't a 0:24:01.960 --> 0:24:05.520 virtual reality headset or an augmented reality headset. It was 0:24:05.600 --> 0:24:08.480 kind of a like I said, a remote viewfinder. But 0:24:08.600 --> 0:24:12.240 still I consider that to be the earliest head mounted display, 0:24:12.400 --> 0:24:16.119 not the sort of Damocles. However, Sutherland would end up 0:24:16.119 --> 0:24:19.240 going on to make lots of other contributions in computer 0:24:19.280 --> 0:24:23.639 graphics as well as the overall concepts that would guide 0:24:23.640 --> 0:24:27.560 both virtual reality and augmented reality development over the next 0:24:27.600 --> 0:24:31.639 several decades. But now it'll be time for me to 0:24:31.720 --> 0:24:34.560 kind of move away from Sutherland to talk about some 0:24:34.640 --> 0:24:37.919 other developments that were important in a R. And before 0:24:37.920 --> 0:24:41.280 I get to that, let's take a quick break to 0:24:41.400 --> 0:24:52.280 thank our sponsor. All Right, we just left off with 0:24:52.359 --> 0:24:57.520 Ivan Sutherland. Now let's talk about a different father of 0:24:57.800 --> 0:25:03.719 augmented reality, Myra Krueger or Doctor Myron Krueger. In nineteen 0:25:03.840 --> 0:25:07.520 seventy four, Dr Krueger created an augmented reality lab called 0:25:07.760 --> 0:25:12.640 video Place. Uh, he was really into this idea of 0:25:12.680 --> 0:25:17.719 seeing the interaction of technology and people in artistic ways. 0:25:18.240 --> 0:25:23.480 He really wanted to explore artistic expressions using technology and 0:25:23.600 --> 0:25:27.880