WEBVTT - TechStuff Classic: Augmenting Your Reality 0:00:04.440 --> 0:00:12.360 Welcome to tech Stuff, a production from iHeartRadio. Hey there, 0:00:12.400 --> 0:00:15.920 and welcome to tech Stuff. I'm your host, Jonathan Strickland. 0:00:15.960 --> 0:00:19.799 I'm an executive producer with iHeartRadio, and how the tech 0:00:19.920 --> 0:00:23.160 are you? It's time for a tech Stuff classic episode. 0:00:23.239 --> 0:00:27.880 This episode is called Augmenting Your Reality and originally published 0:00:27.920 --> 0:00:35.199 November twenty third, twenty sixteen. Obviously, augmented reality is still 0:00:35.280 --> 0:00:39.120 a developing technology. I would argue that no one has 0:00:39.200 --> 0:00:44.240 really created the killer hardware for augmented reality as of yet. 0:00:44.479 --> 0:00:47.400 By the time you listen to this, is entirely possible 0:00:47.479 --> 0:00:52.760 that Apple has already unveiled its mixed reality headset, and 0:00:53.000 --> 0:00:57.000 maybe Apple will be the company that really succeeds where 0:00:57.040 --> 0:01:01.720 others just haven't yet. I don't know. I'm recording this 0:01:02.160 --> 0:01:04.880 way back in May twenty twenty three before Apple has 0:01:04.920 --> 0:01:07.960 held that event, So when it comes round, I guess 0:01:08.000 --> 0:01:12.440 I'll find out. Anyway, that's going down a long tangent. 0:01:12.560 --> 0:01:16.640 Let's listen to this classic episode, Augmenting Your Reality, which 0:01:16.680 --> 0:01:22.120 published November twenty third, twenty sixteen. So I thought I 0:01:22.120 --> 0:01:26.160 would do a deeper dive, a bigger explanation about what 0:01:26.200 --> 0:01:29.680 augmented reality is. What it's all about how it works 0:01:30.120 --> 0:01:35.000 and sort of the applications we might put a R 0:01:35.160 --> 0:01:39.040 toward things that, you know, was it good for tons 0:01:39.080 --> 0:01:42.560 of stuff? As it turns out, So the first thing 0:01:42.600 --> 0:01:46.119 we should do is probably defined some terms, because if 0:01:46.120 --> 0:01:49.840 you haven't really looked into augmented reality and you aren't 0:01:49.880 --> 0:01:53.760 familiar with AR, you might just be lost. I'm going 0:01:53.840 --> 0:01:56.760 to define it all for you right now, because that's 0:01:56.800 --> 0:01:59.880 the kind of stand up guy I am. Technically speaking, 0:02:00.040 --> 0:02:05.120 augmented reality is using digital information to enhance or augment 0:02:05.760 --> 0:02:10.880 and experience in our physical real world. So the way 0:02:10.919 --> 0:02:15.040 we usually see this implemented involves some sort of display 0:02:15.760 --> 0:02:18.320 that has an image of the real world on it 0:02:18.360 --> 0:02:21.400 and it overlays digital information on top of that image. 0:02:21.440 --> 0:02:25.400 So think of like a camera's viewfinder, like an LCD 0:02:25.840 --> 0:02:29.400 screen on a camera, and it actually labels the buildings 0:02:29.440 --> 0:02:31.840 that are in view. When you're out on the street 0:02:31.880 --> 0:02:35.160 and you hold the camera up, or a smartphone or 0:02:35.200 --> 0:02:38.800 even a wearable device like a head mounted display that 0:02:38.840 --> 0:02:41.720 you can look through so you can see the real world. 0:02:41.760 --> 0:02:44.320 You're not just staring at a screen, or if you 0:02:44.360 --> 0:02:46.880 are staring at a screen, you're staring at a video 0:02:47.000 --> 0:02:50.240 feed that is provided by an external camera mounted just 0:02:50.320 --> 0:02:51.920 on the other side of the screen, so it's like 0:02:51.960 --> 0:02:55.240 you're looking through a display in the first place, but 0:02:55.320 --> 0:02:58.920 then on top of that view you have this digital information. 0:02:59.520 --> 0:03:02.400 That's the most common implementation we talk about, but it's 0:03:02.480 --> 0:03:05.080 not the only one. Augmented reality does not have to 0:03:05.200 --> 0:03:10.280 only be or even involve visual information at all. You 0:03:10.320 --> 0:03:14.440 could have audio only augmented reality, for example, But the 0:03:14.480 --> 0:03:17.920 whole idea is that it's something that is created digitally 0:03:18.320 --> 0:03:21.800 to enhance your experience in the real world. Now we 0:03:21.880 --> 0:03:27.720 can contrast this with the concept of virtual reality. Virtual reality, 0:03:27.760 --> 0:03:31.320 of course, is a term where you create an experience 0:03:31.480 --> 0:03:36.760 completely through computer generated means a computer is making all 0:03:36.800 --> 0:03:40.800 the things you see and hear, and maybe even beyond 0:03:40.880 --> 0:03:44.840 that if you have really sophisticated setups, so you might 0:03:44.880 --> 0:03:48.040 have some haptic feedback. Haptic refers to your sense of touch, 0:03:48.440 --> 0:03:50.600 so if you have haptic feedback, that means you're getting 0:03:51.040 --> 0:03:54.840 information feedback through your sense of touch. A common example 0:03:54.880 --> 0:03:58.800 of this is a rumble pack inside a game controller 0:03:59.000 --> 0:04:01.520 where you fire a gun and a first person shooter 0:04:02.000 --> 0:04:05.960 and your controller rumbles as a result, letting you know 0:04:06.000 --> 0:04:11.760 that you are in fact, unleashing virtual destruction upon all 0:04:11.840 --> 0:04:15.360 you survey. Well, the same thing can be true with 0:04:15.720 --> 0:04:19.839 a virtual reality setup. So virtual reality is all about 0:04:19.839 --> 0:04:25.599 constructing an artificial reality, a simulated reality. Augmented reality is 0:04:25.600 --> 0:04:28.880 all about enhancing the one that we are actually in. 0:04:29.320 --> 0:04:34.240 And then there's also mixed reality. Mixed reality is kind 0:04:34.279 --> 0:04:37.159 of sort of in between the two. You might have 0:04:37.240 --> 0:04:42.360 some physical objects within a room that are also mapped 0:04:42.400 --> 0:04:46.400 to a virtual environment, and then you use something like 0:04:46.400 --> 0:04:49.960 a head melted display to enter the virtual environment. That's 0:04:49.960 --> 0:04:53.240 what looks like you're inside. But you have physical objects 0:04:53.279 --> 0:04:55.640 in the room around you that are also mapped to 0:04:55.680 --> 0:04:58.120 the virtual world, meaning you could pick up this physical 0:04:58.160 --> 0:05:00.919 object and you would see that reflect did within the 0:05:01.000 --> 0:05:03.719 virtual world, where you might pick up a sword and 0:05:03.800 --> 0:05:07.120 shield or move a chair or something along those lines. 0:05:08.000 --> 0:05:12.080 So augmented reality, virtual reality, and mixed reality are all 0:05:12.200 --> 0:05:16.160 kind of interrelated, so much so that their histories also 0:05:16.520 --> 0:05:20.920 are very much interrelated. And there's some people who try 0:05:20.920 --> 0:05:25.760 to collect these different technologies, these different approaches and put 0:05:25.760 --> 0:05:28.719 them under a common umbrella, and they tend to use 0:05:28.760 --> 0:05:33.520 the phrase alternate reality, which is unfortunate because that's also 0:05:33.880 --> 0:05:37.080 ar but Alternate reality is kind of the umbrella for virtual, 0:05:37.400 --> 0:05:41.839 augmented and mixed reality. Now that kind of gives you 0:05:42.000 --> 0:05:45.480 the definition of those basic terms, and it is important 0:05:45.480 --> 0:05:49.039 to understand them because they're becoming more and more important today. 0:05:49.480 --> 0:05:52.640 You are already probably aware of a lot of VR 0:05:52.760 --> 0:05:55.680 headsets that are out there on the market as well 0:05:55.720 --> 0:06:00.560 as VR well, they're kind of like case is that 0:06:00.640 --> 0:06:04.520 you slide your smartphone into, so your smartphone becomes the 0:06:05.080 --> 0:06:08.599 actual display on a VR headset. The headset itself is 0:06:08.600 --> 0:06:13.880 more or less just a head mounted case for your phone. 0:06:13.960 --> 0:06:15.360 We've seen a lot of those come out over the 0:06:15.440 --> 0:06:18.320 last few years. We've also seen a lot of AR 0:06:18.600 --> 0:06:22.600 applications come out, typically for things like iPads and smartphones, 0:06:22.600 --> 0:06:26.599 but we've also seen some hardware come out that for 0:06:26.680 --> 0:06:31.039 wearable devices that falls into the augmented reality category, stuff 0:06:31.080 --> 0:06:33.440 like Google Glass, which i'll talk about more a little 0:06:33.480 --> 0:06:37.640 bit later in this episode. For augmented reality to work 0:06:37.880 --> 0:06:42.800 to get this enhanced experience of reality around you, there 0:06:42.839 --> 0:06:46.000 are a lot of technological components that have to come 0:06:46.040 --> 0:06:50.480 together so that you actually do get an experience that 0:06:50.720 --> 0:06:55.320 is meaningful. You have to have technology that quote unquote 0:06:55.400 --> 0:06:58.560 knows where you are and what you are looking at 0:06:58.720 --> 0:07:01.120 or what you are close to in order to get 0:07:01.120 --> 0:07:04.680 that augmented experience. It wouldn't do me any good if 0:07:04.680 --> 0:07:08.760 I put on an augmented reality headset, for example, and 0:07:08.920 --> 0:07:13.560 stared at let's say a famous painting, and instead of 0:07:13.600 --> 0:07:16.840 getting information about the famous painting, I see an exploded 0:07:16.960 --> 0:07:20.240 view of an car engine. That would make no sense. 0:07:20.680 --> 0:07:23.800 So you have to build in technologies in order for 0:07:23.920 --> 0:07:27.320 the AR to understand what it is you're trying to 0:07:27.360 --> 0:07:31.440 do and to augment that experience, which meant that we 0:07:31.480 --> 0:07:35.200 had to wait a pretty good long time for the 0:07:35.320 --> 0:07:41.560 various technologies that we use to create this relationship to 0:07:41.680 --> 0:07:45.560 mature to a point where it was possible. Sometimes we 0:07:45.600 --> 0:07:47.600 had technologies that would allow us to do it, but 0:07:47.800 --> 0:07:53.360 it required tethering headsets to very large computers, which meant 0:07:53.360 --> 0:07:57.080 that you didn't have really any mobility and it really 0:07:57.160 --> 0:08:02.920 limited the usefulness of the actual application. In other cases, 0:08:03.680 --> 0:08:06.800 you could say things like your head tracking technology was 0:08:06.840 --> 0:08:10.160 absolutely necessary for AR to develop the way it did. 0:08:11.000 --> 0:08:15.160 GPS technology as well. Remember it wasn't that long ago 0:08:15.640 --> 0:08:20.880 that we ordinary mirror mortals didn't have access to really 0:08:21.040 --> 0:08:25.960 accurate GPS information For a very long time that was 0:08:26.080 --> 0:08:31.080 purposefully made less accurate. It was a matter of national defense. 0:08:31.800 --> 0:08:34.480 It wasn't until the nineties that you started to see 0:08:34.520 --> 0:08:40.760 GPS become more accurate. For the basic consumer. Way back 0:08:40.760 --> 0:08:43.960 in the day, you might get accuracy of up to 0:08:44.040 --> 0:08:46.559 around one hundred meters, which is not great if you're 0:08:46.559 --> 0:08:49.200 looking for the next place to make your turn. If 0:08:49.280 --> 0:08:53.240 it's one hundred meters away, that's pretty far. But now 0:08:53.240 --> 0:08:55.880 it's within a few feet, so it's much better. That 0:08:56.000 --> 0:08:58.360 sort of stuff all had to come together in order 0:08:58.400 --> 0:09:04.640 for augmented reality to become viable. I almost said a reality, 0:09:04.679 --> 0:09:08.439 but that just starts to sound redundant at any rate, 0:09:08.520 --> 0:09:13.280 Let's talk about some of these technologies we really need, 0:09:13.320 --> 0:09:19.160 things like gyroscopes, accelerometers. These help devices understand their orientation 0:09:20.240 --> 0:09:23.840 where they are in respect to something else, like are 0:09:23.880 --> 0:09:25.760 they For a smartphone, it might be is it in 0:09:25.840 --> 0:09:29.680 landscape mode or portrait mode? But for a head mounted display, 0:09:30.080 --> 0:09:34.240 it would help give the unit the information it needs 0:09:34.280 --> 0:09:36.480 to know which way you're looking, like are you looking 0:09:36.520 --> 0:09:38.440 to the east or to the west. That kind of 0:09:38.440 --> 0:09:45.760 thing also compasses obviously very important GPS sensors, image recognition software, 0:09:45.800 --> 0:09:48.400 but has become really important so that when you are 0:09:48.440 --> 0:09:53.480 looking at something, the system can actually identify what that is. 0:09:53.800 --> 0:09:56.439 In some cases you can get around this. You can 0:09:56.480 --> 0:09:59.480 design an AR system where let's say you make a 0:09:59.520 --> 0:10:05.440 movie post and the AR application has the movie poster 0:10:05.640 --> 0:10:09.319 animate in some way if you hold up a smartphone 0:10:09.320 --> 0:10:13.360 that's running the appropriate app. So I'm just gonna take 0:10:13.720 --> 0:10:16.080 a movie from my past that does not have an 0:10:16.400 --> 0:10:19.800 AR movie poster associated with it, but one that I 0:10:19.840 --> 0:10:22.080 can talk about as if it were a good example, 0:10:22.440 --> 0:10:24.640 and that has to be Big Trouble in Little China, 0:10:25.840 --> 0:10:28.760 universally declared the best movie that has ever been made. 0:10:29.040 --> 0:10:31.520 So you've got your Big Trouble in Little China poster 0:10:31.679 --> 0:10:34.920 up on the wall, and you hold up your smartphone 0:10:34.960 --> 0:10:38.160 and you activate your Big Trouble in Little China movie 0:10:38.200 --> 0:10:42.360 marketing app and the camera on your phone to text 0:10:42.360 --> 0:10:45.439 the poster. You know the posters there well, The app 0:10:45.480 --> 0:10:49.040 and the poster together are able to construct the augmented 0:10:49.120 --> 0:10:52.840 experience because there have been elements put into the poster 0:10:53.200 --> 0:10:56.000 that the app is looking for, and once the app 0:10:56.040 --> 0:11:00.280 identifies that, like, it sees maybe eight different points on 0:11:00.320 --> 0:11:03.199 the poster and because of the orientation of those points, 0:11:03.600 --> 0:11:06.320 it knows what angle it's at, what height it's at 0:11:06.600 --> 0:11:10.280 in relation to the phone, and can give you on 0:11:10.320 --> 0:11:14.840 your display the augmented reality experience. In this case, it's 0:11:14.960 --> 0:11:19.360 obviously Jack Burton and the pork Chop Express eating a sandwich, because, 0:11:19.400 --> 0:11:23.320 as we know, the most riveting scene in the movie 0:11:23.559 --> 0:11:27.160 unfolds in this way. So that would be kind of 0:11:27.200 --> 0:11:29.679 an augmented reality experience where you didn't have to worry 0:11:29.720 --> 0:11:33.760 about every possible application out in the real world. You 0:11:33.840 --> 0:11:36.880 made it for something very specific, which means in your 0:11:36.880 --> 0:11:41.120 software you can have the camera look quote unquote for 0:11:41.440 --> 0:11:46.479 these particular points of reference and thus create the augmented 0:11:46.559 --> 0:11:49.280 experience in that way. If you want to take that 0:11:49.400 --> 0:11:51.360 and move it to the real world where you can 0:11:51.720 --> 0:11:55.240 see augmented information about just the world around you, it 0:11:55.320 --> 0:11:58.840 becomes way more complicated. You have to have very sophisticated 0:11:58.920 --> 0:12:03.840 image recognition saw so that the camera picks up the images, 0:12:04.080 --> 0:12:07.600 the software processes the information, identifies what those images are, 0:12:07.960 --> 0:12:11.839 and gives you the relevant information. So working with all 0:12:11.920 --> 0:12:15.440 the sensors, augmented reality can make this a possibility. So 0:12:15.520 --> 0:12:19.160 another example, let's say you're out on the street in Atlanta, 0:12:19.280 --> 0:12:22.760 you're here in my hometown Atlanta, Georgia, and you're looking 0:12:22.800 --> 0:12:24.520 at a building and you wonder what it is, and 0:12:24.520 --> 0:12:26.480 you hold up your phone and you've got your little 0:12:26.920 --> 0:12:29.960 map app that allows you to look at a real 0:12:30.040 --> 0:12:33.360 world setting and tells you information about it, and it 0:12:33.400 --> 0:12:35.880 tells you it's the Georgia Aquarium. Well, first of all, 0:12:35.880 --> 0:12:37.839 you would probably know that already because the signage there 0:12:37.880 --> 0:12:41.480 is actually pretty good. But the point being that this 0:12:41.520 --> 0:12:45.160 would be something that would tap into the GPS coordinates 0:12:45.200 --> 0:12:47.280 on your phone, so it would know where your location 0:12:47.559 --> 0:12:50.120 was and help narrow that down. The compass would tell 0:12:50.120 --> 0:12:55.240 it what direction you are facing the camera angle. Also, 0:12:55.480 --> 0:12:59.479 you have some image recognition going on there. The accelerometer 0:12:59.559 --> 0:13:02.720 tells the orientation of the phone itself. All of this 0:13:02.840 --> 0:13:06.480 data together would give the software the information needed for 0:13:06.559 --> 0:13:11.720 it to display the labeled Georgia Aquarium on your phone. 0:13:11.760 --> 0:13:16.280 And it all happens in an instant that's pretty amazing. Typically, 0:13:16.280 --> 0:13:20.359 you also have to have some other method to communicate 0:13:20.440 --> 0:13:24.439 with a larger infrastructure, because we don't have the capability 0:13:24.520 --> 0:13:28.720 of building an enormously powerful computer that has all this 0:13:28.800 --> 0:13:32.120 real world information programmed into it and make it a 0:13:32.280 --> 0:13:36.600 handheld or wearable device. So usually you have to pair 0:13:36.720 --> 0:13:40.480 these devices with some other larger infrastructure. Sometimes it's a 0:13:40.520 --> 0:13:44.480 double handshake. For example, with Google Glass, you would use 0:13:44.480 --> 0:13:47.920 Bluetooth to connect Google Glass to a smartphone. Then the 0:13:47.960 --> 0:13:51.800 smartphone would have the connection to the larger internet through 0:13:51.880 --> 0:14:00.800 your smartphone's cell service provider. So while you're experiencing the 0:14:00.840 --> 0:14:04.400 augmented reality through the Google Glass, it's actually communicating through 0:14:04.400 --> 0:14:06.800 your phone to the infrastructure to get the data it 0:14:06.880 --> 0:14:10.280 needs to show you the information. It's showing you very 0:14:10.280 --> 0:14:12.800 important elements. And all of these components, like I said, 0:14:12.920 --> 0:14:16.720 came together more or less around the same time. Most 0:14:16.760 --> 0:14:20.160 of them were being developed independently of each other, and 0:14:20.160 --> 0:14:23.040 it's just that now we're seeing them all converge. That's 0:14:23.080 --> 0:14:26.960 an old favorite word here at tech stuff converge together 0:14:27.000 --> 0:14:29.920 to create the augmented reality experience and make it possible. 0:14:30.760 --> 0:14:32.880 So how did we get here? How did these different 0:14:32.880 --> 0:14:37.080 elements develop? Well, there are a whole bunch of technology 0:14:37.120 --> 0:14:40.560 pioneers who really create the foundation for augmented reality as 0:14:40.560 --> 0:14:43.560 well as virtual reality and mixed reality. But one that 0:14:43.640 --> 0:14:46.120 I think we really need to concentrate on it first 0:14:46.240 --> 0:14:51.120 is Ivan Sutherland. Now Sutherland was born in Hastings, Nebraska 0:14:51.160 --> 0:14:54.200 in nineteen thirty eight, and as a kid, he was 0:14:54.280 --> 0:15:00.520 fascinated with mathematics, particularly geometry, and also with engineering. He 0:15:00.560 --> 0:15:03.920 began to study and experiment with computers while he was 0:15:03.960 --> 0:15:06.560 in school, and this was at a time where personal 0:15:06.560 --> 0:15:09.240 computers weren't a thing. There were no personal computers at 0:15:09.240 --> 0:15:11.880 this point. Computers were actually pretty rare, and they were huge, 0:15:12.360 --> 0:15:16.440 and in fact, they often would rely upon physical media 0:15:16.520 --> 0:15:21.200 formats like punch cards or paper tape to read a program. 0:15:22.000 --> 0:15:25.840 So you didn't even have a disc or like certainly 0:15:25.840 --> 0:15:29.040 nothing like a USB thumb drive or anything like that. 0:15:29.120 --> 0:15:32.360 You actually had to put physical media into the machine 0:15:32.400 --> 0:15:36.080 for it to read and then execute whatever program you 0:15:36.120 --> 0:15:40.480 had designed for that device. He went to college at 0:15:40.520 --> 0:15:44.000 what is now Carnegie Mellon University on a full scholarship. 0:15:44.200 --> 0:15:46.520 He graduated with a Bachelor of Science degree. He would 0:15:46.560 --> 0:15:48.680 then go on to earn a master's degree at cal 0:15:48.760 --> 0:15:53.360 Tech and a PhD in electrical engineering from MIT. And 0:15:53.560 --> 0:15:58.400 actually his doctoral thesis supervisor was Claude Shannon. And we 0:15:58.520 --> 0:16:01.520 talked about Claude Shannon back in the twenty fourteen episode 0:16:01.520 --> 0:16:05.720 Who is Claude Shannon. We recorded that not too long 0:16:05.800 --> 0:16:09.080 after Shannon's passing, So if you want to hear a 0:16:09.120 --> 0:16:13.120 really interesting story about a pioneer in computer science, you 0:16:13.120 --> 0:16:17.200 should go check out that twenty fourteen episode. Back to Sutherland. 0:16:17.560 --> 0:16:22.040 For his thesis, he created something called Sketchpad and that 0:16:22.200 --> 0:16:25.880 was really, by most accounts, the first computer graphical user 0:16:25.920 --> 0:16:29.920 interface or guy A graphic goal user interface means that 0:16:30.160 --> 0:16:36.440 you interact with the computer through graphics representing various commands 0:16:36.520 --> 0:16:39.640 on the computer. Windows and the Mac operating system are 0:16:39.680 --> 0:16:43.520 both examples of graphical user interfaces, as is the interface 0:16:43.560 --> 0:16:46.400 on your smartphone. If you have a smartphone where you 0:16:46.680 --> 0:16:51.520 choose applications on a screen, that's a graphical user interface. Well, 0:16:51.560 --> 0:16:54.400 Sutherland created what is largely considered to be the first 0:16:54.400 --> 0:16:58.080 one of those. After college, he entered military service and 0:16:58.200 --> 0:17:01.920 he was assigned to the National Seecurity Agency. We have 0:17:02.040 --> 0:17:06.479 great friends there. I assume I'm sure they're listening, because 0:17:06.480 --> 0:17:09.960 they're listening to everything at any rate. He entered the 0:17:10.080 --> 0:17:13.320 NSA as an electrical engineer, and in nineteen sixty four 0:17:13.440 --> 0:17:17.200 he replaced JC. R. Licklider as the head of DARPA's 0:17:17.240 --> 0:17:23.360 Information Processing Techniques Office. Or IPTO. And also by back 0:17:23.400 --> 0:17:28.560 then DARPA wasn't DARPA, it was just ARPA. So this 0:17:28.600 --> 0:17:30.320 is the same group, by the way, that would end 0:17:30.440 --> 0:17:33.080 up doing a lot of work that would form the 0:17:33.280 --> 0:17:36.199 ARPANET a few years later, and the arpainet was the 0:17:36.280 --> 0:17:40.200 predecessor to the Internet in some ways. At least, the 0:17:40.320 --> 0:17:44.320 ARPAET was what ended up being the building blocks for 0:17:44.720 --> 0:17:48.479 the infrastructure that would become the Internet. Now, all of 0:17:48.480 --> 0:17:51.520 that work happened after Sutherland had already departed the organization. 0:17:53.080 --> 0:17:57.320 His work became a fundamental component of both virtual and 0:17:57.359 --> 0:18:00.760 augmented reality. As I mentioned earlier, in nineteen sixty five, 0:18:01.160 --> 0:18:03.560 he wrote a piece and essay. It's very short, it's 0:18:03.640 --> 0:18:05.879 very easy read, and you can find it online. The 0:18:05.880 --> 0:18:09.199 title of the essay is the Ultimate Display. And if 0:18:09.240 --> 0:18:13.280 you ever do any research and virtual reality or augmented reality, 0:18:13.520 --> 0:18:16.440 this essay is going to pop up in your research, 0:18:16.800 --> 0:18:19.040 so go ahead and read it. It's like two pages long, 0:18:19.119 --> 0:18:22.840 so it goes very quickly. In that essay he talked 0:18:22.840 --> 0:18:29.080 about several ideas, including the idealized display, the ultimate display, 0:18:29.240 --> 0:18:34.919 something that would be the furthest you could go with 0:18:35.040 --> 0:18:39.000 display technology. Now, keep in mind in his time. By 0:18:39.040 --> 0:18:41.320 his time, he's still alive, by the way, this time 0:18:41.359 --> 0:18:45.879 in the nineteen sixties, he you know, things were just 0:18:46.040 --> 0:18:50.840 restricted to monitors. You might have a light pen, but 0:18:50.960 --> 0:18:54.280 usually you just use a keyboard. Like it was pretty 0:18:54.880 --> 0:18:57.919 bare bones. But he said, let's push this as far 0:18:58.000 --> 0:19:02.320 as we can imagine it. His example, he thought of 0:19:02.359 --> 0:19:06.400 a room that would be completely controlled by computers. Everything 0:19:06.400 --> 0:19:10.000 you would experience within that room would be generated by 0:19:10.000 --> 0:19:14.919 a computer. Everything you see here, smell, taste, and touch, 0:19:15.800 --> 0:19:18.320 all of it generated by computers. The computer would even 0:19:18.400 --> 0:19:22.640 be able to form physical objects out of pure matter itself. Now, 0:19:22.680 --> 0:19:26.320 he wasn't suggesting that this would ever be a device 0:19:26.359 --> 0:19:28.399 that we would actually be able to build. He was 0:19:28.440 --> 0:19:33.560 just saying, what is the ultimate incarnation of display technology? 0:19:34.359 --> 0:19:37.159 And if you read it, you realize, oh, this is 0:19:37.200 --> 0:19:39.879 where the Star Trek Next Generation writers got their idea 0:19:39.920 --> 0:19:43.360 for the Holidack. But unlike Star Trek the Next Generation, 0:19:43.640 --> 0:19:46.840 the Ultimate Display would not go on the Fritz every 0:19:46.840 --> 0:19:49.840 other episode and try to kill the crew. It was 0:19:49.920 --> 0:19:54.720 better than that. The Ultimate Display was sort of a 0:19:55.840 --> 0:20:01.760 foundational like Philosophically, it was foundational forvirual reality and augmented reality. 0:20:01.840 --> 0:20:05.520 This idea of a very immersive experience where you, as 0:20:05.560 --> 0:20:12.400 a user are surrounded somehow by this computer generated experience. 0:20:12.880 --> 0:20:15.720 And that's true both with augmented reality and virtual reality. 0:20:15.800 --> 0:20:20.600 In augmented reality, the real world is still there, but 0:20:20.680 --> 0:20:24.960 you get this enhanced experience that is completely computer generated. 0:20:26.200 --> 0:20:31.200 So in nineteen sixty eight, Sutherland and a student named 0:20:31.280 --> 0:20:36.679 Danny Cohen would create a VR ar head mounted display 0:20:36.960 --> 0:20:41.560 or HMD, and they nicknamed it the Sword of Damocles. 0:20:42.800 --> 0:20:45.439 Why because you had to suspend it from the ceiling. 0:20:45.480 --> 0:20:48.359 It was too heavy to wear on your head. You 0:20:48.400 --> 0:20:53.520 needed it to be nice and sturdy. It included transparent lenses, 0:20:53.560 --> 0:20:57.400 which meant you could overlay computer information on the lenses themselves, 0:20:57.640 --> 0:21:00.040 and thus you could look through the lenses at the 0:21:00.080 --> 0:21:04.760 real world and have these wireframe graphics on top of 0:21:04.840 --> 0:21:07.199 what you were looking at. And it also had a 0:21:07.240 --> 0:21:11.240 magnetic tracking system, meaning that it had sensors that could 0:21:11.240 --> 0:21:14.320 detect magnetic fields, and as you turned your head or 0:21:14.480 --> 0:21:18.399 you change the inclination of your head, it would change 0:21:18.440 --> 0:21:21.600 the magnetic field and this would be relayed as a 0:21:21.680 --> 0:21:26.720 command to the visual center the actual lenses themselves, so 0:21:26.760 --> 0:21:30.040 that it would the change would be reflected in what 0:21:30.080 --> 0:21:33.240 you saw. So if you have a virtual environment and 0:21:33.280 --> 0:21:35.359 you turn your head to the left, you want the 0:21:35.480 --> 0:21:38.040 view within the virtual environment to go to the left too, 0:21:38.520 --> 0:21:41.480 But without head tracking technology that's impossible. So this was 0:21:41.520 --> 0:21:44.400 a very early example of head tracking technology, and again 0:21:44.440 --> 0:21:49.720 it used magnets magnetic fields in order to do that. Obviously, 0:21:49.720 --> 0:21:53.720 it's also really important for augmented reality. Again, if the 0:21:54.160 --> 0:21:57.399 AR system doesn't detect that you are looking around, then 0:21:57.480 --> 0:22:01.240 you're not getting relevant information for the specific thing you 0:22:01.359 --> 0:22:05.439 are looking at. Anyway, as I said, the graphics were 0:22:05.440 --> 0:22:09.400 pretty primitive, they were wireframe drawings, but they still showed 0:22:09.520 --> 0:22:15.280 that this was a viable approach to technology using HMD 0:22:15.440 --> 0:22:18.840 for augmented or virtual reality use. Oh and one other 0:22:18.880 --> 0:22:21.399 note I should make So a lot of people say 0:22:21.440 --> 0:22:24.160 the sort of Damocles was the first head mounted display, 0:22:24.840 --> 0:22:26.840 and they say, you know, this is the first HMD 0:22:27.040 --> 0:22:30.479 was made in nineteen sixty eight. I take issue with that. 0:22:31.359 --> 0:22:34.520 I don't think of the sort of Damocles as the 0:22:34.560 --> 0:22:37.800 first head mounted display. That to me should go to 0:22:38.080 --> 0:22:44.840 a different invention called the head site hadsight now that 0:22:44.920 --> 0:22:48.800 was developed by phil Co and unlike the sort of damocles. 0:22:49.400 --> 0:22:53.800 It didn't create a virtual world. Instead, the head site 0:22:54.040 --> 0:22:58.479 was sort of a remote viewfinder for a video camera. 0:23:00.000 --> 0:23:04.240 Imagine that you got a camera mounted on a mechanical 0:23:05.320 --> 0:23:09.199 swiveling mount, so you can move it left right, you 0:23:09.200 --> 0:23:13.000 can change the orientation the inclination as well, and then 0:23:13.040 --> 0:23:15.879 you have that mapped to a head mounted display, so 0:23:15.920 --> 0:23:18.200 that if I put the display on and I look 0:23:18.280 --> 0:23:20.880 to the left, the camera pans to the left. If 0:23:20.880 --> 0:23:22.440 I look to the right, it pans to the right. 0:23:22.560 --> 0:23:26.000 That sort of thing. It was meant to be a 0:23:26.040 --> 0:23:29.879 way for people to operate a camera in a remote 0:23:29.960 --> 0:23:33.800 location that might not be very friendly to a human 0:23:33.880 --> 0:23:37.639 being standing there. For example, the exterior of an aircraft. 0:23:38.000 --> 0:23:40.080 You could have a camera mounted on the outside of 0:23:40.080 --> 0:23:43.639 your aircraft that would allow an engineer on the inside 0:23:44.080 --> 0:23:48.560 to look around and maybe help a pilot land or 0:23:48.800 --> 0:23:52.360 navigate in a dangerous situation, or just get an idea 0:23:52.520 --> 0:23:56.959 of the status of the aircraft itself. This was very 0:23:57.080 --> 0:24:01.560 much a technology that was being pushed by the military, 0:24:02.560 --> 0:24:06.440 an idea to create more military uses using this technology 0:24:06.480 --> 0:24:11.040 to make the military more competent, more adept at very 0:24:11.240 --> 0:24:17.160 rapid changing situations on the technology front, so headsite preceded 0:24:17.520 --> 0:24:20.320 the sort of Damocles by about seven years. It came 0:24:20.359 --> 0:24:23.000 out around nineteen sixty one. But again it wasn't a 0:24:23.080 --> 0:24:26.639 virtual reality headset or an augmented reality headset. It was 0:24:26.720 --> 0:24:29.600 kind of a like I said, a remote viewfinder. But 0:24:29.720 --> 0:24:33.360 still I consider that to be the earliest head mounted display, 0:24:33.560 --> 0:24:37.199 not the sort of Damocles. However, Sutherland would end up 0:24:37.200 --> 0:24:40.320 going on to make lots of other contributions in computer 0:24:40.400 --> 0:24:44.720 graphics as well as the overall concepts that would guide 0:24:44.760 --> 0:24:48.680 both virtual reality and augmented reality development over the next 0:24:48.680 --> 0:24:52.760 several decades. But now it'll be time for me to 0:24:52.840 --> 0:24:55.639 kind of move away from Sutherland and talk about some 0:24:55.720 --> 0:24:59.080 other developments that were important in AR. And before I 0:24:59.119 --> 0:25:02.760 get to that, let's take a quick break to thank 0:25:02.800 --> 0:25:15.879 our sponsor. All Right, we just left off with Ivan Sutherland. 0:25:15.960 --> 0:25:21.679 Now let's talk about a different father of augmented reality, 0:25:22.160 --> 0:25:26.920 Myron Krueger or doctor Myron Krueger. In nineteen seventy four, 0:25:26.960 --> 0:25:31.119 doctor Krueger created an augmented reality lab called Video Place. 0:25:32.440 --> 0:25:36.920 He was really into this idea of seeing the interaction 0:25:37.119 --> 0:25:41.560 of technology and people in artistic ways. He really wanted 0:25:41.600 --> 0:25:47.640 to explore artistic expressions using technology and people working together. 0:25:48.960 --> 0:25:51.600 So he wanted to create an artificial reality environment that 0:25:51.680 --> 0:25:55.080 didn't require the user to wear special equipment. You wouldn't 0:25:55.119 --> 0:25:57.560 have to put on a head Moulton display, or wear 0:25:57.920 --> 0:26:02.560 special gloves, or use in any kind of device to 0:26:02.640 --> 0:26:06.760 control your actions, because that's a barrier between you and 0:26:06.800 --> 0:26:12.679 the experience. Instead, his version consisted of a laboratory that 0:26:12.760 --> 0:26:17.920