WEBVTT - Augmenting Your Reality 0:00:04.160 --> 0:00:07.160 Get in text with technology with tex Stuff from how 0:00:07.240 --> 0:00:14.160 stuff works dot com. Hey there, and welcome to tex Stuff. 0:00:14.200 --> 0:00:17.680 I am your host, Jonathan Strickland, and today I'm going 0:00:17.760 --> 0:00:22.000 to augment your reality, or at least tell you what 0:00:22.320 --> 0:00:24.720 augmented reality and a R is all about. And if 0:00:24.800 --> 0:00:28.040 this seems like it's deja vu, that means you are 0:00:28.240 --> 0:00:33.600 a long time listener of tech stuff, because we did 0:00:33.840 --> 0:00:38.120 our episode about a are way back in two thousand nine. 0:00:38.159 --> 0:00:40.839 That's when Chris Palette was my co host and together 0:00:40.880 --> 0:00:43.879 we recorded an episode that we called What in the 0:00:43.920 --> 0:00:47.640 World Is Augmented Reality? And that was a pretty short 0:00:47.680 --> 0:00:50.720 overview of the concept, like twenty minutes long, and that 0:00:50.800 --> 0:00:55.840 included two different listener males. So I thought I would 0:00:55.840 --> 0:00:59.960 do a deeper dive, a bigger explanation about what augment 0:01:00.040 --> 0:01:03.200 the reality is, what it's all about, how it works, 0:01:03.560 --> 0:01:08.400 and sort of the applications we might put a R 0:01:08.560 --> 0:01:12.520 toward things that, you know, was it good for tons 0:01:12.520 --> 0:01:16.000 of stuff? As it turns out, So the first thing 0:01:16.040 --> 0:01:19.520 we should do, it's probably defined some terms, because if 0:01:19.560 --> 0:01:23.280 you haven't really looked into augmented reality and you aren't 0:01:23.319 --> 0:01:27.039 familiar with a R, you might just be lost. I'm 0:01:27.040 --> 0:01:30.240 gonna define it all for you right now, because that's 0:01:30.240 --> 0:01:33.360 the kind of stand up guy I am. Technically speaking, 0:01:33.400 --> 0:01:38.520 augmented reality is using digital information to enhance or augment 0:01:39.200 --> 0:01:44.319 and experience in our physical real world. So the way 0:01:44.360 --> 0:01:49.080 we usually see this implemented involves some sort of display 0:01:49.200 --> 0:01:51.720 that has an image of the real world on it, 0:01:51.800 --> 0:01:54.800 and it overlays digital information on top of that image. 0:01:54.880 --> 0:01:58.840 So think of like a camera's viewfinder, like an LCD 0:01:59.280 --> 0:02:02.840 screen on a camera, and it actually labels the buildings 0:02:02.880 --> 0:02:05.240 that are in view. When you're out on the street 0:02:05.320 --> 0:02:08.560 and you hold the camera up, or a smartphone or 0:02:08.600 --> 0:02:12.200 even a wearable device like a head mounted display that 0:02:12.280 --> 0:02:15.160 you can look through so you can see the real world. 0:02:15.200 --> 0:02:17.760 You're not just staring at a screen, or if you 0:02:17.800 --> 0:02:20.320 are staring at a screen, you're staring at a video 0:02:20.440 --> 0:02:23.640 feed that is provided by an external camera amounted just 0:02:23.720 --> 0:02:25.320 on the other side of the screen. So it's like 0:02:25.400 --> 0:02:28.680 you're looking through a display in the first place, but 0:02:28.760 --> 0:02:32.440 then on top of that view you have this digital information. 0:02:33.000 --> 0:02:35.880 That's the most common implementation we talk about, but it's 0:02:35.880 --> 0:02:38.520 not the only one. Augmented reality does not have to 0:02:38.639 --> 0:02:43.760 only be or even involve visual information at all. You 0:02:43.760 --> 0:02:47.880 could have audio only augmented reality, for example, But the 0:02:47.919 --> 0:02:51.360 whole idea is that it's something that has created digitally 0:02:51.720 --> 0:02:55.280 to enhance your experience in the real world. Now we 0:02:55.320 --> 0:03:01.160 can contrast this with the concept of virtual reality. Virtual reality, 0:03:01.200 --> 0:03:04.760 of course, is a term where you create an experience 0:03:04.880 --> 0:03:10.200 completely through computer generated means a computer is making all 0:03:10.240 --> 0:03:14.240 the things you see and here, and maybe even beyond 0:03:14.280 --> 0:03:18.079 that if you have really sophisticated UH setups, so you 0:03:18.160 --> 0:03:21.040 might have some haptic feedback. Haptic refers to your sense 0:03:21.080 --> 0:03:23.520 of touch, so if you have haptic feedback, that means 0:03:23.560 --> 0:03:27.840 you're getting information feedback through your sense of touch. Common 0:03:27.880 --> 0:03:32.400 example of this is a rumble pack inside a game controller, 0:03:32.440 --> 0:03:34.320 where you know you fire a gun and a first 0:03:34.360 --> 0:03:39.080 person shooter and your controller rumbles as a result, letting 0:03:39.080 --> 0:03:44.440 you know that you are in fact, unleashing virtual destruction 0:03:44.600 --> 0:03:48.320 upon all you survey. Well, the same thing can be 0:03:48.360 --> 0:03:52.880 true with virtual reality set up. So virtual reality is 0:03:52.920 --> 0:03:58.400 all about constructing an artificial reality a simulated reality. Augmented 0:03:58.440 --> 0:04:01.000 reality is all about enhance and seeing the one that 0:04:01.080 --> 0:04:05.840 we are actually in. And then there's also mixed reality. 0:04:06.240 --> 0:04:09.800 Mixed reality is kind of sort of in between the two. 0:04:09.880 --> 0:04:14.560 You might have some physical objects within a room that 0:04:14.760 --> 0:04:18.640 are also mapped to a virtual environment, and then you 0:04:18.800 --> 0:04:22.200 use something like a head molded display to enter the 0:04:22.279 --> 0:04:25.760 virtual environment. That's what looks like you're inside. But you 0:04:25.800 --> 0:04:28.160 have physical objects in the room around you that are 0:04:28.200 --> 0:04:30.680 also mapped to the virtual world, Meaning you could pick 0:04:30.800 --> 0:04:33.720 up this physical object and you would see that reflected 0:04:34.000 --> 0:04:36.719 within the virtual world, where you might pick up a 0:04:36.760 --> 0:04:40.000 sword and shield or move a chair or something along 0:04:40.040 --> 0:04:44.920 those lines. So augmented reality, virtual reality, and mixed reality 0:04:45.000 --> 0:04:48.120 are all kind of inter related, so much so that 0:04:48.160 --> 0:04:53.279 their histories also are very much inter related. And there's 0:04:53.279 --> 0:04:56.920 some people who try to collect these different technologies, these 0:04:56.960 --> 0:05:00.880 different approaches and put them under a co an umbrella, 0:05:00.920 --> 0:05:05.520 and they tend to use the phrase alternate reality, which 0:05:05.520 --> 0:05:09.039 is unfortunate because that's also a r but alternate reality 0:05:09.080 --> 0:05:12.440 is kind of the umbrella for virtual, augmented and mixed reality. 0:05:13.680 --> 0:05:16.680 Uh that that that kind of gives you the definition 0:05:16.720 --> 0:05:19.640 of those basic terms, and it is important to understand 0:05:19.680 --> 0:05:23.120 them because they're becoming more and more important today. You 0:05:23.279 --> 0:05:27.280 are already probably aware of a lot of VR headsets 0:05:27.320 --> 0:05:29.760 that are out there on the market as well as VR. 0:05:30.640 --> 0:05:34.279 Uh Well, they're they're kind of like cases that you 0:05:34.320 --> 0:05:38.920 slide your smartphone into, so your smartphone becomes the actual 0:05:39.320 --> 0:05:42.160 display on a VR headset. The headset itself is more 0:05:42.240 --> 0:05:46.200 or less just a head mounted case for your phone. 0:05:47.400 --> 0:05:48.800 We've seen a lot of those come out over the 0:05:48.880 --> 0:05:51.479 last few years. We've also seen a lot of a 0:05:51.760 --> 0:05:56.000 R applications come out, typically for things like iPads and smartphones, 0:05:56.080 --> 0:06:00.040 but we've also seen some hardware come out that for 0:06:00.160 --> 0:06:04.479 wearable devices that falls into the augmented reality category, stuff 0:06:04.480 --> 0:06:06.920 like Google Glass, which I'll talk about more a little 0:06:06.920 --> 0:06:11.039 bit later in this episode. For augmented reality to work 0:06:11.279 --> 0:06:16.240 to get this enhanced experience of reality around you, there 0:06:16.240 --> 0:06:19.440 are a lot of technological components that have to come 0:06:19.480 --> 0:06:24.839 together so that you actually do get experience that is meaningful. 0:06:25.800 --> 0:06:28.719 You know, you have to have technology that quote unquote 0:06:28.800 --> 0:06:32.000 knows where you are and what you are looking at 0:06:32.120 --> 0:06:34.560 or what you are close to in order to get 0:06:34.560 --> 0:06:38.080 that augmented experience. It wouldn't do me any good if 0:06:38.120 --> 0:06:42.200 I put on an augmented reality headset, for example, and 0:06:42.360 --> 0:06:47.000 stared at let's say, a famous painting, and instead of 0:06:47.040 --> 0:06:50.280 getting information about the famous painting I see an exploded 0:06:50.400 --> 0:06:53.720 view of an car engine That would make no sense. 0:06:54.120 --> 0:06:57.240 So you have to build in technologies in order for 0:06:57.360 --> 0:07:00.599 the a R to understand what it is you're trying 0:07:00.600 --> 0:07:04.760 to do and to augment that experience, which meant that 0:07:04.800 --> 0:07:08.240 we had to wait a pretty good long time for 0:07:08.560 --> 0:07:13.640 the various technologies that we used to to create this 0:07:13.800 --> 0:07:17.920 relationship to mature to a point where it was possible. 0:07:18.320 --> 0:07:20.920 Sometimes we had technologies that would allow us to do it, 0:07:20.960 --> 0:07:26.400 but it required uh tethering headsets to very large computers, 0:07:26.400 --> 0:07:29.520 which meant that you didn't have really any mobility, and 0:07:29.800 --> 0:07:34.760 uh it really limited the usefulness of the actual application. 0:07:35.520 --> 0:07:38.960 In other cases, you could say things like your head 0:07:38.960 --> 0:07:42.920 tracking technology was absolutely necessary for a R to develop 0:07:42.960 --> 0:07:47.480 the way it did. GPS technology as well. Remember it 0:07:47.520 --> 0:07:52.559 wasn't that long ago that we ordinary mere mortals didn't 0:07:52.640 --> 0:07:57.320 have access to really accurate GPS information. For a very 0:07:57.360 --> 0:08:02.720 long time that was purposely made less accurate. It was 0:08:02.760 --> 0:08:07.000 a matter of national defense. It wasn't until the nineties 0:08:07.040 --> 0:08:10.840 that you started to ce GPS become more accurate for 0:08:10.960 --> 0:08:15.080 the basic consumer. Wait, back in the day, you might 0:08:15.160 --> 0:08:18.800 get accuracy of up to around a hundred meters, which 0:08:18.840 --> 0:08:21.080 is not great if you're looking for the next place 0:08:21.120 --> 0:08:24.000 to make your turn. If it's a hundred meters away, 0:08:24.040 --> 0:08:27.760 that's that's pretty far. But now it's within a few feet, 0:08:27.880 --> 0:08:30.600 so it's much better. That sort of stuff all had 0:08:30.640 --> 0:08:33.959 to come together in order for augmented reality to become 0:08:34.600 --> 0:08:38.600 a viable I almost said a reality, but that just 0:08:38.640 --> 0:08:43.040 starts to sound redundant at any rate. Let's talk about 0:08:43.080 --> 0:08:46.960 some of these technologies we were. We really need things 0:08:46.960 --> 0:08:53.959 like gyroscopes, accelerometers. These help devices understand their orientation where 0:08:54.000 --> 0:08:57.480 they are in respect to something else, like are they 0:08:57.840 --> 0:08:59.760 For a smartphone, it might be is it in landscape 0:08:59.760 --> 0:09:03.439 mode or portrait mode? But for a head mounted display, 0:09:03.520 --> 0:09:07.440 it would help give the the unit the information it 0:09:07.440 --> 0:09:09.480 needs to know which way you're looking, like are you 0:09:09.600 --> 0:09:11.760 looking to the east or to the west. That kind 0:09:11.760 --> 0:09:17.520 of thing. UH also compasses obviously very important GPS sensors, 0:09:17.640 --> 0:09:20.920 image recognition software, but has become really important so that 0:09:21.280 --> 0:09:25.320 when you are looking at something UH, the system can 0:09:25.360 --> 0:09:28.480 actually identify what that is. In some cases you can 0:09:28.480 --> 0:09:31.840 get around this. You can design an a R system 0:09:31.880 --> 0:09:34.880 where let's say you make a movie poster and the 0:09:34.960 --> 0:09:40.000 a R application has the movie poster animate in some 0:09:40.080 --> 0:09:43.480 way if you hold up a smartphone that's running the 0:09:43.520 --> 0:09:47.880 appropriate app. So I'm just gonna take a movie from 0:09:47.880 --> 0:09:50.600 my past that does not have an a R movie 0:09:50.640 --> 0:09:53.600 poster associated with it, but one that I can talk 0:09:53.679 --> 0:09:56.160 about as if it were a good example, and that 0:09:56.280 --> 0:10:00.360 has to be Big Trouble in Little China, universally cleared 0:10:00.400 --> 0:10:02.800 the best movie that has ever been made. So you've 0:10:02.840 --> 0:10:05.360 got your Big Trouble and Little China poster up on 0:10:05.400 --> 0:10:08.679 the wall, and you hold up your smartphone and you 0:10:08.720 --> 0:10:12.520 activate your Big Trouble and Little China movie marketing app, 0:10:13.320 --> 0:10:16.440 and the camera on your phone detect the poster. It's 0:10:16.640 --> 0:10:19.120 you know, the posters there. Well. The app and the 0:10:19.160 --> 0:10:23.600 poster together are able to construct the augmented experience because 0:10:24.120 --> 0:10:26.840 there have been elements put into the poster that the 0:10:26.880 --> 0:10:30.840 app is looking for. And once the app identifies that, like, 0:10:30.920 --> 0:10:34.440 it sees maybe eight different points on the poster, and 0:10:34.520 --> 0:10:37.720 because of the orientation of those points, it knows what 0:10:37.920 --> 0:10:40.760 angle it's at, what height it's at in relation to 0:10:40.800 --> 0:10:44.960 the phone, and can give you on your display the 0:10:45.040 --> 0:10:49.560 augmented reality experience. In this case, it's obviously Jack Burton 0:10:49.600 --> 0:10:53.040 and the Pork Chop Express eating a sandwich, because, as 0:10:53.040 --> 0:10:57.559 we know, the most riveting scene in the movie unfolds 0:10:57.559 --> 0:11:00.720 in this way. So that would be kind of an 0:11:00.760 --> 0:11:04.000 augmented reality experience where you didn't have to worry about 0:11:04.280 --> 0:11:07.440 every possible application out in the real world. You made 0:11:07.480 --> 0:11:10.840 it for something very specific, which means in your software 0:11:10.880 --> 0:11:15.280 you can have the camera look quote unquote for these 0:11:15.559 --> 0:11:20.560 particular points of reference and thus create the augmented experience 0:11:20.559 --> 0:11:22.960 in that way. If you want to take that and 0:11:23.000 --> 0:11:25.439 move it to the real world where you can see 0:11:25.480 --> 0:11:29.120 augmented information about just the world around you, it becomes 0:11:29.200 --> 0:11:32.680 way more complicated. You have to have very sophisticated image 0:11:32.679 --> 0:11:37.280 recognition software so that the camera picks up the images, 0:11:37.520 --> 0:11:41.040 the software processes the information, identifies what those images are, 0:11:41.400 --> 0:11:45.280 and gives you the relevant information. So working with all 0:11:45.320 --> 0:11:48.880 the sensors, augmented reality can make this a possibility. So 0:11:48.920 --> 0:11:52.400 another example, let's say you're out on the street in Atlanta. 0:11:52.720 --> 0:11:56.240 You're here in my hometown, Atlanta, Georgia, and you're looking 0:11:56.240 --> 0:11:57.960 at a building and you wonder what it is, and 0:11:57.960 --> 0:11:59.920 you hold up your phone and you've got your little 0:12:00.360 --> 0:12:03.400 map app that allows you to look at a real 0:12:03.440 --> 0:12:06.800 world setting and tells you information about it, and it 0:12:06.840 --> 0:12:09.280 tells you it's the Georgia Aquarium. Well, first of all, 0:12:09.320 --> 0:12:11.280 you would probably know that already because the signage there 0:12:11.320 --> 0:12:14.920 is actually pretty good. But the point being that this 0:12:14.960 --> 0:12:18.040 would be uh something that would tap into the GPS 0:12:18.080 --> 0:12:20.319 coordinates on your phone, so it would know where your 0:12:20.360 --> 0:12:23.440 location was and help narrow that down. The compass would 0:12:23.440 --> 0:12:28.680 tell it what direction you are facing the camera angle. Also, 0:12:28.760 --> 0:12:32.160 when you have some image recognition going on there, the 0:12:32.200 --> 0:12:36.040 accelerometer tells the orientation of the phone itself. All of 0:12:36.080 --> 0:12:39.440 this data together would give the software the information needed 0:12:39.840 --> 0:12:45.160 for it to display the label Georgia Aquarium on your phone. 0:12:45.200 --> 0:12:48.040 And it all happens in an instant that's pretty amazing. 0:12:49.280 --> 0:12:52.480 Typically you also have to have some other method to 0:12:53.000 --> 0:12:57.160 communicate with a larger infrastructure, because we don't have the 0:12:57.200 --> 0:13:01.640 capability of building an enormous, sleep powerful computer that has 0:13:01.760 --> 0:13:05.240 all this real world information programmed into it and make 0:13:05.280 --> 0:13:09.480 it a handheld or wearable device. So usually you have 0:13:09.600 --> 0:13:13.680 to pair these devices with some other larger infrastructure. Sometimes 0:13:13.679 --> 0:13:17.439 it's a double handshake. For example, with Google Glass, you 0:13:17.480 --> 0:13:20.720 would use Bluetooth to connect Google Glass to a smartphone. 0:13:21.080 --> 0:13:24.200 Then the smartphone would have the connection to the larger 0:13:24.240 --> 0:13:30.920 internet through your smartphones UH cell service provider. So while 0:13:30.960 --> 0:13:36.280 you're experiencing the augmented reality through the Google Glass, it's 0:13:36.280 --> 0:13:39.640 actually communicating through your phone to the infrastructure to get 0:13:39.679 --> 0:13:42.120 the data it needs to show you the information. It's 0:13:42.120 --> 0:13:45.680 showing you very important elements. And all of these components, 0:13:45.760 --> 0:13:48.400 like I said, came together more or less around the 0:13:48.480 --> 0:13:52.640 same time. Most of them were being developed independently of 0:13:52.679 --> 0:13:54.920 each other, and it's just that now we're seeing them 0:13:54.920 --> 0:13:58.520 all converge. That's an old favorite word here at tech 0:13:58.559 --> 0:14:02.599 stuff converge together to create the augmented reality experience and 0:14:02.640 --> 0:14:05.480 make it possible. So how did we get here? How 0:14:05.559 --> 0:14:09.560 did these different elements develop? Well, there are a whole 0:14:09.559 --> 0:14:12.720 bunch of technology pioneers who really created the foundation for 0:14:12.760 --> 0:14:15.960 augmented reality as well as virtual reality and mixed reality. 0:14:16.400 --> 0:14:18.800 But one that I think we really need to concentrate 0:14:18.840 --> 0:14:23.080 on at first is Ivan Sutherland. Now. Sutherland was born 0:14:23.160 --> 0:14:26.880 in Hastings, Nebraska in nineteen thirty eight, and as a 0:14:26.960 --> 0:14:32.240 kid he was fascinated with mathematics, particularly geometry, and also 0:14:32.280 --> 0:14:36.520 with engineering. He began to study and experiment with computers 0:14:36.680 --> 0:14:39.240 while he was in school, and this was at a 0:14:39.240 --> 0:14:41.640 time where personal computers weren't a thing. There were no 0:14:41.720 --> 0:14:44.440 personal computers at this point. Computers were actually pretty rare 0:14:44.480 --> 0:14:47.960 and they were huge, and in fact, they often would 0:14:48.000 --> 0:14:52.200 rely upon physical media formats like punch punch cards or 0:14:52.280 --> 0:14:57.280 paper tape to read a program. So you didn't even 0:14:57.280 --> 0:15:00.200 have a disk or like certainly nothing like a US 0:15:00.200 --> 0:15:03.240 Beat thumb drive or anything like that. You you actually 0:15:03.240 --> 0:15:06.080 had to put physical media into the machine for it 0:15:06.120 --> 0:15:10.200 to read and then execute whatever program you had designed 0:15:10.200 --> 0:15:14.320 for that device. He went to college at what is 0:15:14.360 --> 0:15:18.320 now Carnegie Mellon University on a full scholarship. He graduated 0:15:18.320 --> 0:15:20.280 with a Bachelor of Science degree. He would then go 0:15:20.320 --> 0:15:22.680 on to earn a master's degree at cal Tech and 0:15:22.800 --> 0:15:26.880 a PhD in electrical engineering from m I T. And 0:15:27.000 --> 0:15:31.840 actually his doctoral thesis supervisor was Claude Shannon, And we 0:15:31.920 --> 0:15:34.040 talked about Claude Shannon back in the two thousand and 0:15:34.080 --> 0:15:38.040 fourteen episode who is Claude Shannon? Um. We recorded that 0:15:38.120 --> 0:15:41.680 not too long after Shannon's passing. So if you want 0:15:41.680 --> 0:15:45.280 to hear a really interesting story about a pioneer in 0:15:45.280 --> 0:15:47.920 computer science, you should go check out that two thousand 0:15:47.920 --> 0:15:52.240 and fourteen episode. Back to Sutherland. For his thesis, he 0:15:52.360 --> 0:15:56.400 created something called sketch Pad and that was really, by 0:15:56.440 --> 0:16:00.920 most accounts, the first computer graphical user interface or gooey. 0:16:00.960 --> 0:16:05.160 A graphic goal user interface means that you interact with 0:16:05.280 --> 0:16:10.640 the computer through graphics representing various commands on the computer. 0:16:10.760 --> 0:16:13.960 Windows and the Mac operating system are both examples of 0:16:13.960 --> 0:16:17.800 graphical user interfaces, as is the interface on your smartphone. 0:16:17.880 --> 0:16:21.800 If you have a smartphone where you choose applications on 0:16:21.840 --> 0:16:25.960 a screen, that's a graphical user interface. Well, Sutherland created 0:16:26.000 --> 0:16:28.520 what is largely considered to be the first one of those. 0:16:29.080 --> 0:16:32.520 After college, he entered military service and he was assigned 0:16:32.560 --> 0:16:36.480 to the National Security Agency. We have great friends there. 0:16:37.320 --> 0:16:40.600 I assume I'm sure they're listening, because they're listening to 0:16:40.680 --> 0:16:43.800 everything at any rate. He entered the n s A 0:16:43.880 --> 0:16:47.000 as an electrical engineer, and in nineteen sixty four he 0:16:47.080 --> 0:16:50.040 replaced J. C. R. Lick Lighter as the head of 0:16:50.120 --> 0:16:56.440 DARPA's Information Processing Techniques Office, or TIPTOE. And also, by 0:16:56.560 --> 0:17:01.520 back then, DARPA wasn't DARPA, it was just ARPA. Uh uh, 0:17:01.640 --> 0:17:03.400 So this is the same group, by the way, that 0:17:03.400 --> 0:17:05.879 would end up doing a lot of work that would 0:17:05.920 --> 0:17:09.400 form the Arpanette a few years later, and the Arpanette 0:17:09.440 --> 0:17:13.560 was the predecessor to the Internet in some ways. At least, 0:17:13.600 --> 0:17:16.840 the Arpanet was what ended up being the the building 0:17:16.920 --> 0:17:21.560 blocks for the infrastructure that would become the Internet. Now, 0:17:21.600 --> 0:17:24.200 all of that work happened after Sutherland had already departed 0:17:24.200 --> 0:17:30.160 the organization. His work became a fundamental component of both 0:17:30.240 --> 0:17:33.280 virtual and augmented reality. As I mentioned earlier, in nine 0:17:34.600 --> 0:17:36.959 he wrote a piece and essay. It's very short, it's 0:17:37.080 --> 0:17:39.280 very easy read, and you can find it online. The 0:17:39.320 --> 0:17:42.640 title of the essay is the Ultimate Display. And if 0:17:42.680 --> 0:17:46.720 you ever do any research in virtual reality or augmented reality, 0:17:46.960 --> 0:17:49.880 this essay is going to pop up in your research, 0:17:50.240 --> 0:17:52.480 so go ahead and read it. It's like two pages long, 0:17:52.560 --> 0:17:56.240 so it goes very quickly. In that essay he talked 0:17:56.280 --> 0:18:02.520 about several ideas, including the idealized display, the ultimate display, 0:18:02.720 --> 0:18:06.760 something that would be the the furthest you could go 0:18:08.240 --> 0:18:11.040 with display technology. Now, keep in mind, in his time, 0:18:12.280 --> 0:18:13.879 by his time he's still alive, by the way, but 0:18:14.320 --> 0:18:17.240 this time in the nineties sixties, he uh, you know, 0:18:17.400 --> 0:18:22.879 things were just restricted to monitors. You might have a 0:18:22.960 --> 0:18:26.879 light pen, but usually you just use a keyboard. Like 0:18:27.119 --> 0:18:30.919 it was pretty bare bones. They said, let's push this 0:18:31.040 --> 0:18:35.119 as far as we can imagine it. And in his example, 0:18:35.240 --> 0:18:38.200 he thought of a room that would be completely controlled 0:18:38.240 --> 0:18:42.520 by computers. Everything you would experience within that room would 0:18:42.520 --> 0:18:47.600 be generated by a computer. Everything you see here, smell, taste, 0:18:47.720 --> 0:18:51.320 and touch, all of it generated by computers. The computer 0:18:51.359 --> 0:18:54.359 would even be able to form physical objects out of 0:18:54.400 --> 0:18:58.280 pure matter itself. Now, he wasn't suggesting that this would 0:18:58.320 --> 0:19:01.160 ever be a device that we would actually be able 0:19:01.200 --> 0:19:03.960 to build. He was just saying, what is the ultimate 0:19:04.600 --> 0:19:10.280 incarnation of display technology? And if you read it, you realize, oh, 0:19:10.320 --> 0:19:12.840 this is where the Star Trek Next Generation writers got 0:19:12.840 --> 0:19:16.119 their idea for the Holidack. But unlike Star Trek the 0:19:16.160 --> 0:19:19.439 Next Generation, the Ultimate Display would not go on the 0:19:19.480 --> 0:19:22.000 Fritz every other episode and try to kill the crew. 0:19:23.040 --> 0:19:27.920 It was better than that. The Ultimate Display was sort 0:19:27.920 --> 0:19:32.600 of a a foundational like Philosophically, it was foundational for 0:19:33.320 --> 0:19:36.240 virtual reality and augmented reality. This idea of a very 0:19:36.280 --> 0:19:42.000 immersive experience where you, as a user, are surrounded somehow 0:19:42.320 --> 0:19:47.240 by this computer generated experience. And that's true both with 0:19:47.280 --> 0:19:52.200 augmented reality and virtual reality. And augmented reality, the real 0:19:52.240 --> 0:19:56.000 world is still there, but you get this enhanced experience 0:19:56.040 --> 0:20:02.040 that is completely computer generated. So in nineteen sixty eight, 0:20:02.680 --> 0:20:07.399 Sutherland and a student named Danny Cohen would create a VR, 0:20:07.840 --> 0:20:12.640 a r head mounted display or h MD, and they 0:20:12.760 --> 0:20:17.440 nicknamed it the Sword of Damocles. Why because you had 0:20:17.480 --> 0:20:19.680 to suspend it from the ceiling. It was too heavy 0:20:19.720 --> 0:20:22.439 to wear on your head. You needed it to be 0:20:22.560 --> 0:20:27.440 nice and sturdy. Uh. It included transparent lenses, which meant 0:20:27.480 --> 0:20:31.199 you could overlay computer information on the lenses themselves, and 0:20:31.240 --> 0:20:33.680 thus you could look through the lenses at the real 0:20:33.720 --> 0:20:38.239 world and have these wire framed graphics on top of 0:20:38.240 --> 0:20:40.639 what you were looking at. And it also had a 0:20:40.680 --> 0:20:44.640 magnetic tracking system, meaning that it had sensors that could 0:20:44.640 --> 0:20:47.760 detect magnetic fields, and as you turned your head or 0:20:47.920 --> 0:20:51.840 you change the inclination of your head, it would change 0:20:51.880 --> 0:20:55.000 the magnetic field and this would be relayed as a 0:20:55.119 --> 0:21:00.119 command to the visual center the actual lenses themselves, so 0:21:00.240 --> 0:21:03.439 that it would the change would be reflected in what 0:21:03.520 --> 0:21:06.679 you saw. So if you have a virtual environment and 0:21:06.720 --> 0:21:08.760 you turn your head to the left, you want the 0:21:08.880 --> 0:21:11.639 view within the virtual environment to go to the left too, 0:21:11.920 --> 0:21:14.919 but without head tracking technology that's impossible. So this was 0:21:14.960 --> 0:21:17.880 a very early example of head tracking technology. And again 0:21:17.880 --> 0:21:22.080 it used magnets magnetic fields in order to do that. Uh. 0:21:22.680 --> 0:21:26.760 Obviously it's also really important for augmented reality. Again, if 0:21:27.080 --> 0:21:30.160 the a R system doesn't detect that you are looking around, 0:21:30.680 --> 0:21:34.400 then you're not getting relevant information, not for the specific 0:21:34.400 --> 0:21:38.399 thing you are looking at. Anyway, as I said, the 0:21:38.400 --> 0:21:41.080 graphics were pretty primitive. There were wire frame drawings, but 0:21:41.119 --> 0:21:46.960 they still showed that this was a viable approach to 0:21:47.040 --> 0:21:51.760 technology using HMD for augmented or virtual reality use. Oh 0:21:51.800 --> 0:21:54.120 and one other note I should make so a lot 0:21:54.160 --> 0:21:56.520 of people say the sort of Damocles was the first 0:21:56.520 --> 0:21:59.320 head mounted display, and they say, you know, this is 0:21:59.359 --> 0:22:03.080 the first m D was made in nineteen I take 0:22:03.200 --> 0:22:06.760 issue with that. I don't think of the sort of 0:22:06.840 --> 0:22:10.359 Damocles as the first head mounted display. That to me 0:22:10.560 --> 0:22:14.119 should go to a different invention called the head site 0:22:15.240 --> 0:22:18.120 h G A D S I G h T now 0:22:18.160 --> 0:22:22.280 that was developed by Philco and unlike the sort of Damocles, 0:22:22.800 --> 0:22:27.200 it didn't create a virtual world. Instead, the head site 0:22:27.480 --> 0:22:31.919 was sort of a remote viewfinder for a video camera. 0:22:32.960 --> 0:22:36.920 So imagine that you've got a camera mounted on a 0:22:36.960 --> 0:22:42.520 mechanical uh swiveling mount, so you can move it left right, 0:22:42.560 --> 0:22:46.239 you can change the orientation the inclination as well, and 0:22:46.280 --> 0:22:49.120 then you have that mapped to a head mounted display, 0:22:49.200 --> 0:22:51.440 so that if I put the display on and I 0:22:51.480 --> 0:22:53.960 looked to the left, the camera pans to the left. 0:22:54.160 --> 0:22:55.880 If I look to the right, it pans to the right. 0:22:56.000 --> 0:22:59.720 That sort of thing it was meant to be away 0:22:59.800 --> 0:23:03.360 for for people to operate a camera in a remote 0:23:03.400 --> 0:23:07.240 location that might not be very friendly to a human 0:23:07.280 --> 0:23:11.040 being standing there. For example, the exterior of an aircraft. 0:23:11.440 --> 0:23:13.520 You could have a camera mounted on the outside of 0:23:13.520 --> 0:23:17.080 your aircraft that would allow an engineer on the inside 0:23:17.520 --> 0:23:22.000 to look around and maybe help a pilot land or 0:23:22.200 --> 0:23:25.840 navigate in a dangerous situation, or just get an idea 0:23:26.000 --> 0:23:30.440 of the status of the aircraft itself. This was very 0:23:30.520 --> 0:23:35.120 much a technology that was being pushed by the military, 0:23:36.000 --> 0:23:39.879 an idea to create more military uses using this technology 0:23:39.920 --> 0:23:44.560 to make the military more competent, more adept at very 0:23:44.680 --> 0:23:50.520 rapid changing situations on the technology front, So headsite preceded 0:23:50.960 --> 0:23:53.760 the sort of damocles by about seven years. That came 0:23:53.760 --> 0:23:56.439 out around nineteen sixty one, but again it wasn't a 0:23:56.480 --> 0:24:00.320 virtual reality headset or an augmented reality headset. Was kind 0:24:00.320 --> 0:24:03.479 of a like I said, a remote viewfinder. But still 0:24:03.800 --> 0:24:06.760 I consider that to be the earliest head mounted display, 0:24:06.920 --> 0:24:10.639 not the sort of damocles. However, Sutherland would end up 0:24:10.640 --> 0:24:13.760 going on to make lots of other contributions in computer 0:24:13.800 --> 0:24:18.200 graphics as well as the overall concepts that would guide 0:24:18.200 --> 0:24:22.119 both virtual reality and augmented reality development over the next 0:24:22.119 --> 0:24:26.199 several decades. But now it'll be time for me to 0:24:26.280 --> 0:24:29.080 kind of move away from Sutherland to talk about some 0:24:29.160 --> 0:24:32.440 other developments that were important in a R. And before 0:24:32.440 --> 0:24:35.800 I get to that, let's take a quick break to 0:24:35.960 --> 0:24:46.800 thank our sponsor. All Right, we just left off with 0:24:46.920 --> 0:24:52.040 Ivan Sutherland. Now let's talk about a different father of 0:24:52.359 --> 0:24:58.879 augmented reality, Myron Krueger or Dr Myron Krueger. In nineteen four, 0:24:59.000 --> 0:25:04.440 Dr Krueger create an augmented reality lab called Video Place. Uh. 0:25:04.480 --> 0:25:09.000 He was really into this idea of seeing the interaction 0:25:09.119 --> 0:25:13.560 of technology and people in artistic ways. He really wanted 0:25:13.600 --> 0:25:19.640 to explore artistic expressions using technology and people working together. 0:25:20.960 --> 0:25:23.600 So he wanted to create an artificial reality environment that 0:25:23.680 --> 0:25:27.080 didn't require the user to wear special equipment. You wouldn't 0:25:27.080 --> 0:25:29.600 have to put on a head mold display, or wear 0:25:29.960 --> 0:25:35.080