WEBVTT - Zoom and Enhance 0:00:00.160 --> 0:00:07.280 Brought to you by Toyota. Let's go places. Welcome to 0:00:07.440 --> 0:00:16.400 Forward Thinking Either and Welcome to Forward Thinking, the podcast 0:00:16.440 --> 0:00:19.600 that looks at the future and says zoom manned enhance. 0:00:19.920 --> 0:00:23.200 I'm Jonathan Strickland, I'm Lauren Malcolm, and I'm Joe McCormick, 0:00:23.360 --> 0:00:26.040 and we wanted to talk about this idea of playing 0:00:26.040 --> 0:00:29.160 with cameras and the images that they take, whether it's 0:00:29.200 --> 0:00:32.200 still photography or video images, and this idea of being 0:00:32.240 --> 0:00:36.599 able to manipulate those images and maybe even do this 0:00:36.680 --> 0:00:39.120 crazy zoom and enhanced thing. You guys, You guys are 0:00:39.159 --> 0:00:42.400 familiar with the TV, right, I need some help you guys, 0:00:42.920 --> 0:00:48.519 what why is that there's been a theft? What was stolen? Well, 0:00:48.920 --> 0:00:52.680 somebody broke into my home and they stole all of 0:00:52.760 --> 0:00:57.000 my VHS tapes of the Super Mario Show. Wow, that's 0:00:57.120 --> 0:01:00.040 all of them. That's not necessarily a loss, but I 0:01:00.040 --> 0:01:02.920 appreciate that you're hurting Lauren. No, this is a loss 0:01:02.960 --> 0:01:06.600 not just for Joe over all mankind. It was the 0:01:06.640 --> 0:01:10.520 Super Mario Brothers Super Show and now I don't have it, 0:01:10.959 --> 0:01:15.720 but I've got a lead. Okay, So I had a 0:01:15.840 --> 0:01:19.400 digital security camera installed in my home, not too, as 0:01:19.400 --> 0:01:22.280 long as you have a VHS collection of Super Mario. 0:01:22.440 --> 0:01:25.800 But you have a digital security camera digital security camera, 0:01:26.200 --> 0:01:32.399 and it it records video in still frames of Oh, 0:01:32.440 --> 0:01:36.960 I guess they're about a hundred pixels each, so very 0:01:37.000 --> 0:01:43.720 low resolution. That's a hundred pixels, I mean. Well, anyway, 0:01:43.800 --> 0:01:46.240 so I got together puzzles that had a hundred pieces, 0:01:46.800 --> 0:01:49.120 so it's not that much. And it takes a picture 0:01:49.200 --> 0:01:51.960 every sixty seconds, so that's not bad. I don't call 0:01:52.000 --> 0:01:54.560 that video. I call that a series of very crappy 0:01:54.640 --> 0:01:57.919 still photos. But I have a photo of the thief, okay, 0:01:57.960 --> 0:02:00.120 all right, and I'm going to try to find him. 0:02:00.120 --> 0:02:02.280 It could also be sasquatch. We don't know you, but 0:02:02.360 --> 0:02:05.600 I need your help, okay, to identify this person. Right. 0:02:05.720 --> 0:02:07.800 So do we I know here that we got a 0:02:07.840 --> 0:02:10.400 lot of techie people here in the office. Do we 0:02:10.440 --> 0:02:13.519 have one of those enhanced computers? I can just scan 0:02:13.680 --> 0:02:16.880 this in, right, and and we can press enhance and 0:02:16.960 --> 0:02:19.880 it will give us the dudes face. No, we don't 0:02:19.919 --> 0:02:21.760 have one of those. Why don't we have They've got 0:02:21.760 --> 0:02:25.400 to be pretty cheap by now, right, No, So alright, 0:02:25.440 --> 0:02:29.960 so what what we're referring to? Here's this trope. I 0:02:30.160 --> 0:02:32.240 first of all, Joe, I certainly hope that that was 0:02:32.919 --> 0:02:35.360 just an example story that you haven't actually suffered a 0:02:35.440 --> 0:02:37.959 terrible loss of that. Just kidding, all right, good, So 0:02:37.960 --> 0:02:41.560 so the Super Mario videotapes are safe. Okay, good. So 0:02:41.760 --> 0:02:44.000 what we're talking about is a television trope. In fact, 0:02:44.320 --> 0:02:47.560 one of my favorite websites to just waste time on 0:02:47.720 --> 0:02:50.040 is one of the TV tropes sites where you can 0:02:50.080 --> 0:02:52.960 just read about all these these sort of cliches that 0:02:53.000 --> 0:02:56.359 have been used in television and film for about as 0:02:56.360 --> 0:03:00.320 long as the those types of media have existed. Really um, 0:03:00.360 --> 0:03:02.040 and some of them are a little newer than others. 0:03:02.240 --> 0:03:05.959 Zoom and enhanced, pretty new because we didn't have those 0:03:06.040 --> 0:03:08.040 kind of we didn't even think about those kind of 0:03:08.080 --> 0:03:11.200 capabilities until maybe the last couple of decades. Okay, so 0:03:11.400 --> 0:03:13.320 it may have been earlier than this. But the earliest 0:03:13.360 --> 0:03:17.160 example I can think of is in N two the 0:03:17.200 --> 0:03:21.960 movie Blade Runner Ridley Scott Harrison Ford um based on 0:03:22.000 --> 0:03:25.040 the Philip K. Dick. But there's a scene in it 0:03:25.320 --> 0:03:29.600 where Harrison Ford's character is doing some investigating. Yeah, he's 0:03:29.600 --> 0:03:32.480 got a photograph and he's got it on his computer 0:03:32.560 --> 0:03:36.720 screen and he keeps shuffling around, looking at different frames 0:03:36.720 --> 0:03:40.720 of the photograph and then zooming way in and enhancing 0:03:40.760 --> 0:03:46.120 the photo. Yeah, it's um, it's it's the way TV 0:03:46.200 --> 0:03:49.200 tropes defines it. They call it the enhanced button, very 0:03:49.240 --> 0:03:51.240 similar to the scenario you were talking about the very 0:03:51.280 --> 0:03:54.080 top of the show, Joe, He says, a staple of 0:03:54.120 --> 0:03:57.200 any crime drama. The enhanced button on the computer is 0:03:57.240 --> 0:04:00.080 able to turn a tiny, blurred, grainy image in a 0:04:00.120 --> 0:04:03.840 photo or video into a clear, unmistakable piece of evidence. 0:04:04.120 --> 0:04:08.880 This process is virtually instantaneous, unless added dramatic tension is required, 0:04:08.920 --> 0:04:13.920 in which case extra technobabble or more applied flabottanum may 0:04:13.920 --> 0:04:16.640 be needed. These, by the way, or other tropes may 0:04:16.680 --> 0:04:20.200 require someone to stand next to the computer intoning enhance, 0:04:21.200 --> 0:04:25.240 enhance for full effect. So yeah, this idea of taking 0:04:25.320 --> 0:04:29.440 something like an image a photograph that is imperfect, it 0:04:29.480 --> 0:04:32.080 has a limited amount of data in it, and then 0:04:32.200 --> 0:04:35.520 enhancing it so that it becomes something useful, something recognizable, 0:04:36.560 --> 0:04:38.880 is that a thing? Is that possible? All right? At 0:04:38.920 --> 0:04:42.760 least the way it's done in Hollywood? Not so much. Well, 0:04:43.040 --> 0:04:45.920 it's not possible the way they do it because in 0:04:46.000 --> 0:04:50.400 these movies and TV shows. You're just getting information that 0:04:50.560 --> 0:04:53.600 just blatantly just was not there before, right, Right, you 0:04:53.800 --> 0:04:57.479 can only enhance up to as much information as actually 0:04:57.520 --> 0:05:00.560 exists in the physical record of that infra exactly. You 0:05:00.640 --> 0:05:04.960 can't extrapolate new things whole cloth. Right The way, the 0:05:05.000 --> 0:05:07.080 way I'd put it is that there's sort of a 0:05:07.120 --> 0:05:10.760 bottleneck on data at the moment an image is captured. 0:05:11.400 --> 0:05:15.640 Um data comes into the camera and you record a 0:05:15.640 --> 0:05:18.280 certain amount of it, and then data comes out later. Now, 0:05:18.320 --> 0:05:21.920 you can manipulate that later data in all kinds of ways, 0:05:22.120 --> 0:05:24.720 but you can't ever put in more than came in 0:05:24.800 --> 0:05:27.680 through the lens to begin with, well, not not directly 0:05:27.920 --> 0:05:31.600 onto the raw image file. You can manipulate it enough 0:05:31.720 --> 0:05:35.360 and through guesswork. Really you're you're kind of you're kind 0:05:35.360 --> 0:05:38.839 of making some assumptions where you can fill in information 0:05:38.880 --> 0:05:42.480 that is missing. But it's not like you are uncovering 0:05:42.720 --> 0:05:45.800 the information that was there. You're actually creating new stuff 0:05:45.839 --> 0:05:48.520 to go in with the stuff that was captured at 0:05:48.560 --> 0:05:50.479 that moment. Right. It would be like if you saw, 0:05:50.640 --> 0:05:54.599 for example, somebody's shadow in a photograph. Right now, if 0:05:54.640 --> 0:05:56.960 somebody was a really good I don't know, this is 0:05:57.000 --> 0:05:59.719 a real thing, but it was really good at identifying humans. 0:05:59.760 --> 0:06:02.720 But the shadows they cast, they can do the same 0:06:02.760 --> 0:06:04.400 kind of thing. They can look at the shadow and 0:06:04.440 --> 0:06:07.520 the picture and say, oh, it's probably this person, But 0:06:07.680 --> 0:06:11.760 you still wouldn't actually have that person's image somewhere hidden 0:06:11.880 --> 0:06:15.440 in the file right. And you know, also, like if 0:06:15.480 --> 0:06:17.360 you've got a picture of the back of a person's head, 0:06:17.520 --> 0:06:21.320 it's not like the image of their face is contained 0:06:21.320 --> 0:06:24.360 in that data somewhere, and you can't just just rotate 0:06:24.440 --> 0:06:26.680 person and and have you know, there's no button to 0:06:26.720 --> 0:06:28.880 do that. It's amazing how often you can do that 0:06:28.920 --> 0:06:31.000 in science fiction though, like turn it around. I want 0:06:31.000 --> 0:06:33.400 to see this. I want to see this still image 0:06:33.440 --> 0:06:36.320 from the opposite angle, as if we can magically place 0:06:36.440 --> 0:06:39.880 the camera anywhere after the effect. Uh, what what you 0:06:39.920 --> 0:06:42.120 can do in a virtual environment? I mean, there are 0:06:42.360 --> 0:06:44.400 camera tricks you can do in a virtual environment that 0:06:44.480 --> 0:06:47.760 completely defy all laws of physics because that they aren't 0:06:47.839 --> 0:06:51.039 a problem in a virtual environment. So, for example, in 0:06:51.080 --> 0:06:53.279 a video game, there are a lot of video games 0:06:53.279 --> 0:06:55.800 where you can capture the footage of you playing while 0:06:55.839 --> 0:06:57.640 you're playing the video game, and then watch it later 0:06:57.800 --> 0:06:59.960 and you can even watch it from various camera angle 0:07:00.040 --> 0:07:02.279 is depending upon the type of game. Like, some games 0:07:02.320 --> 0:07:05.159 give you essentially free reign. You can place the camera 0:07:05.279 --> 0:07:07.520 anywhere you like. You can have a free roaming camera 0:07:07.560 --> 0:07:10.880 and move it dynamically as the scene plays back. So 0:07:10.960 --> 0:07:14.200 even though while you were playing you had one set 0:07:14.240 --> 0:07:16.320 of perceptions, you know, you might have been able to 0:07:16.320 --> 0:07:18.320 move the camera around then too, but you were limited 0:07:18.320 --> 0:07:20.480 at that time by whatever was happening right then and there. 0:07:20.840 --> 0:07:24.160 But in playback you may be given unlimited freedom. Now 0:07:24.200 --> 0:07:26.360 that's just not the case with real life. I mean, 0:07:26.720 --> 0:07:29.160 obviously you know it's that that that data exists in 0:07:29.200 --> 0:07:31.840 real life, but if there's nothing there to record it 0:07:31.920 --> 0:07:37.320 right then, So when I was talking about the you know, 0:07:37.480 --> 0:07:40.239 creating stuff so that you can fill in the gaps, 0:07:40.400 --> 0:07:44.480 it's where you're you're not again, you're not uncovering lost information. 0:07:44.880 --> 0:07:48.560 You're not or or obvious skated information. That's the way 0:07:48.560 --> 0:07:52.400 it comes across in these television and film examples where 0:07:52.560 --> 0:07:54.920 the answer is hidden in the file and you just 0:07:54.960 --> 0:07:56.800 have to dig it out. Yeah. Yeah, there's just a 0:07:56.840 --> 0:07:59.480 button on your computer that somehow makes it go from 0:07:59.480 --> 0:08:01.960 blurry to not blurry, and that it was the info 0:08:02.080 --> 0:08:04.640 was always there, it just needed to be not blurry. 0:08:04.680 --> 0:08:07.760 That's not the case. That's not the way it works. Um. 0:08:07.880 --> 0:08:10.520 There was a great article and Wired that talked about 0:08:10.560 --> 0:08:15.600 this and talked about some approaches that that various technicians 0:08:15.600 --> 0:08:20.120 have made to really uh address this issue. You know, 0:08:20.160 --> 0:08:24.000 I was talking about creating data to fill in those gaps, 0:08:24.040 --> 0:08:26.960 the stuff that's not actually there in these photos. You know, 0:08:27.400 --> 0:08:29.680 in the Hollywood and TV versions, it always seems like 0:08:29.960 --> 0:08:32.679 the information is there, it's just blurry or whatever. And 0:08:32.679 --> 0:08:35.520 when you hit this button, it removes whatever that problem 0:08:35.559 --> 0:08:37.480 was and you get to look at the information that 0:08:37.559 --> 0:08:40.280 was always there. That's not the case, right. We're talking 0:08:40.280 --> 0:08:43.480 about photos and video where stuff is missing. But you 0:08:43.559 --> 0:08:46.000 can start to fill in some of those gaps by 0:08:46.040 --> 0:08:48.439 creating stuff, by making guesses. And there was a great 0:08:48.559 --> 0:08:51.200 article and Wired that talked about this and talked about 0:08:51.320 --> 0:08:55.440 compressed sensing and sparsity. These are concepts that are used 0:08:55.440 --> 0:08:58.240 by technicians to kind of fill in information that might 0:08:58.280 --> 0:09:00.920 be missing, either a file. You know, maybe maybe the 0:09:00.960 --> 0:09:03.520 image is just really fuzzy. It could be an old photograph, 0:09:03.600 --> 0:09:05.920 or it could just have been made with a poor camera, 0:09:06.120 --> 0:09:08.680 or maybe something was on the lens or whatever, and 0:09:08.720 --> 0:09:12.320 we can use these techniques to try and fill in information. Now, 0:09:12.320 --> 0:09:16.000 in the Wired article, they had a really interesting analogy. 0:09:16.040 --> 0:09:19.160 They said that, imagine that you have a book where 0:09:19.320 --> 0:09:23.440 on one page of the book you have almost half 0:09:23.520 --> 0:09:26.280 or maybe even more of the information missing. So you've 0:09:26.280 --> 0:09:29.640 got just missing words in sentences. Now on the pages before, 0:09:29.640 --> 0:09:32.480 in the pages after, you've got some information there, But 0:09:32.800 --> 0:09:35.720 on the page that you're interested in your missing words. 0:09:36.200 --> 0:09:38.800 It would be like trying to extrapolate what those missing 0:09:38.800 --> 0:09:41.360 words were just based on the little bits of information 0:09:41.400 --> 0:09:44.520 you had on the previous and following pages. It's really 0:09:44.559 --> 0:09:49.160 difficult to do. However, with compressed sensing, there was an 0:09:49.200 --> 0:09:52.800 interesting development. There was a guy named Emmanuel Condez who 0:09:52.920 --> 0:09:58.120 was looking at what an image called the shep logan phantom. Now, 0:09:58.160 --> 0:10:00.520 this is actually an image that technicians you us in 0:10:00.600 --> 0:10:03.720 order to test imaging algorithms, and it kind of looks 0:10:03.760 --> 0:10:08.040 like a an alien with a slightly raised eyebrow, kind 0:10:08.080 --> 0:10:11.280 of a snarky alien. Now, what what he did was 0:10:11.320 --> 0:10:14.880 he took an older, fuzzy version of the image, not 0:10:14.920 --> 0:10:17.600 an older one, but a fuzzy one. So he was 0:10:17.640 --> 0:10:21.760 testing an algorithm, trying to see if a particular technique 0:10:22.320 --> 0:10:26.320 would allow him to uh to sharpen this image up 0:10:26.360 --> 0:10:28.640 a little bit. And the way the technique works is 0:10:28.679 --> 0:10:32.480 it looks for the simplest approach to filling out the 0:10:32.520 --> 0:10:37.160 information that's missing by look it's samples pixels in the image, 0:10:37.600 --> 0:10:40.000 and then from those samples, it starts to create simple 0:10:40.080 --> 0:10:43.280 shapes that are their color matched to the various parts 0:10:43.360 --> 0:10:47.000 of that sampled image. Now it tries to use the 0:10:47.000 --> 0:10:51.080 fewest number of these shapes to fill out this this 0:10:51.200 --> 0:10:54.160 photo or this this picture doesn't have to just be 0:10:54.200 --> 0:10:55.960 a picture. By the way, you can actually apply the 0:10:56.040 --> 0:10:59.280 same technique to other kinds of media, including music, where 0:10:59.320 --> 0:11:01.920 you know you might have a low sample rate for 0:11:02.000 --> 0:11:03.840 an old MP three and you want to try and 0:11:03.920 --> 0:11:05.719 enhance it. It could do the same sort of thing 0:11:05.760 --> 0:11:07.720 with that right right, Basically, anything with a with a 0:11:07.760 --> 0:11:10.920 wave form is going to operate under very similar principles. 0:11:10.960 --> 0:11:15.640 So what they're doing here is the reason why he 0:11:15.679 --> 0:11:17.160 was doing this in the first place was not just 0:11:17.200 --> 0:11:20.480 to get a sharper picture of an alien. The idea 0:11:20.559 --> 0:11:23.640 was to try and enhance m r I images because 0:11:24.160 --> 0:11:26.760 the best way to get a very very clear mr 0:11:26.800 --> 0:11:29.440 I image is to put someone in an MRI machine 0:11:29.440 --> 0:11:32.640 and have them stay perfectly still for a couple of minutes. 0:11:33.080 --> 0:11:35.680 But to stay perfectly still for a couple of minutes 0:11:35.760 --> 0:11:38.560 usually means having to put them under anesthesia. So they 0:11:38.600 --> 0:11:43.120 actually stopped breathing. That's how still they needed people to be. Now, 0:11:43.160 --> 0:11:46.960 that's not really that great an option. So he was 0:11:47.000 --> 0:11:49.720 looking at what if I took this approach to try 0:11:49.720 --> 0:11:52.600 and take an image that was captured in say forty 0:11:52.679 --> 0:11:56.280 seconds as opposed to two minutes, and then try to 0:11:56.360 --> 0:11:58.720 use this this uh, this simple technique to see if 0:11:58.720 --> 0:12:01.439 I can sharpen it up. So he runs the the 0:12:01.520 --> 0:12:05.600 image through this algorithm he's created and turned out that 0:12:05.679 --> 0:12:10.120 the resulting image was a perfect match two or a 0:12:10.200 --> 0:12:15.320 near perfect match to the UH the original version of 0:12:15.360 --> 0:12:18.600 the shep Logan phantom image. And he thought, well, that's weird. 0:12:19.040 --> 0:12:22.040 That can't be right. There's no way that worked. And 0:12:22.080 --> 0:12:24.120 so he tested it again and it got the same 0:12:24.160 --> 0:12:26.800 result and ended up showing it off to some other 0:12:26.880 --> 0:12:29.560 folks and and they really began to put their heads 0:12:29.559 --> 0:12:31.480 together and wrote a white paper on it and a 0:12:31.559 --> 0:12:35.640 research paper all about this UH technique, and um, yeah, 0:12:35.720 --> 0:12:38.319 it takes about a hundred thousand pixels for example, and 0:12:38.440 --> 0:12:42.240 just really focus on those and build out these shapes 0:12:42.920 --> 0:12:48.199 and it could build a usable image. But um, a 0:12:48.280 --> 0:12:50.559 couple of caveats. One is that it can take a 0:12:50.640 --> 0:12:54.079 few hours to do this as the algorithm goes through 0:12:54.200 --> 0:12:57.679 all the different variations of the simplest way of approaching this, 0:12:58.280 --> 0:13:01.160 And there is a chance that the resulting image that 0:13:01.200 --> 0:13:03.600 you get back at the end is not a match 0:13:03.720 --> 0:13:06.560 for what it should have been. There is that chance. 0:13:06.640 --> 0:13:10.400 It's a small chance according to the researchers, but it 0:13:10.440 --> 0:13:15.840 can happen because the computer is just basically guessing based 0:13:15.880 --> 0:13:18.760 upon all the little bit of information that it has. 0:13:19.679 --> 0:13:23.640 So in that sense, if you have this this area 0:13:23.880 --> 0:13:26.960 of doubt where you know, you might say, well, you know, 0:13:27.000 --> 0:13:29.240 according to our computer model, this is what the image 0:13:29.280 --> 0:13:31.400 would have looked like if we had looked at it 0:13:31.520 --> 0:13:35.000 at this scale. Uh, you you know, you have to 0:13:35.080 --> 0:13:37.080 keep that in mind. You have to remember, like, this 0:13:37.160 --> 0:13:41.199 is what it probably looks like, not this is definitely 0:13:41.240 --> 0:13:44.560 the image. So that's another difference from the Hollywood version. 0:13:44.640 --> 0:13:46.600 Right with the Hollywood one, as soon as they do 0:13:46.679 --> 0:13:51.000 the enhance, there's our guy, there's no way it's there's 0:13:51.000 --> 0:13:53.120 our guy. It's the guy who was built third in 0:13:53.160 --> 0:13:57.120 the film, so now we know we got him, whereas 0:13:57.160 --> 0:14:00.280 this is just some dude. Yeah, So I mean that's 0:14:00.280 --> 0:14:04.240 an interesting approach. And the whole idea of sparsity is 0:14:04.280 --> 0:14:08.319 this idea of going with the simplest um and fewest 0:14:08.400 --> 0:14:10.840 number of simple shapes. So it might be like it 0:14:10.840 --> 0:14:14.360 it detects a couple of blue pixels, uh in a 0:14:14.480 --> 0:14:16.520 in a or a few blue pixels in an area, 0:14:16.600 --> 0:14:18.000 and then it just fills out the rest of that 0:14:18.040 --> 0:14:21.960 area with the same color blue, so that it's approaching 0:14:22.000 --> 0:14:24.360 at you know, saying well this, this is probably part 0:14:24.440 --> 0:14:26.680 of a border for this thing. I'm just going to 0:14:26.800 --> 0:14:29.760 fill in the rest. And it does that thousands and 0:14:29.800 --> 0:14:32.720 thousands of times for the entire image. But yeah, it's 0:14:32.720 --> 0:14:36.160 still not the kind of instantaneous approach we see in 0:14:36.840 --> 0:14:41.840 popular films and television so well, on a much smaller 0:14:41.880 --> 0:14:45.200 scale and in that kind of instantaneous sort of time frame. 0:14:45.320 --> 0:14:47.120 That's that's a function of photoshop. I mean, you know 0:14:47.160 --> 0:14:49.920 you can you can click that I forget what the 0:14:49.920 --> 0:14:51.480 function is called in there. You can just click that 0:14:51.480 --> 0:14:53.360 little button and have it kind of fill out what 0:14:53.440 --> 0:14:56.040 a line would have looked like, yeah, yeah, there are 0:14:56.240 --> 0:14:58.360 too what's around it. There are a lot of algorithms 0:14:58.400 --> 0:15:01.120 out there that take this approach where it looks at 0:15:01.120 --> 0:15:04.240 the existing data and tries to extrapolate what the rest 0:15:04.280 --> 0:15:06.680 of it should be. Uh, and it's you know there 0:15:06.800 --> 0:15:09.240 and I'm talking about a few pixels, right, and they 0:15:09.280 --> 0:15:13.480 have different degrees of of uh of sophistication and and 0:15:13.640 --> 0:15:16.960 resolution and uh, but nothing on the scale of the 0:15:17.040 --> 0:15:19.720 Hollywood version. Sure, sure, I I do think that that 0:15:19.800 --> 0:15:23.560 what we see in those kind of everyday applications is 0:15:23.560 --> 0:15:26.080 is what leads to some of the confusion about what 0:15:26.240 --> 0:15:29.840 like the professionals can do. Um like a like like 0:15:29.880 --> 0:15:31.880 with Google Maps. You know, you can you can zoom 0:15:31.880 --> 0:15:34.000 and enhance in a Google map, but that's because it's 0:15:34.000 --> 0:15:37.360 built of these multiple tiles of images then, right, that 0:15:37.440 --> 0:15:40.880 that have greater acuity on the lower levels. It's not 0:15:40.920 --> 0:15:45.440 like when you are at you know, at like satellite 0:15:45.560 --> 0:15:48.560 view of the Earth that has all the same detail 0:15:48.680 --> 0:15:50.920 as it would if you are a low flying plane. 0:15:51.320 --> 0:15:54.960 They have different sets of images that are geolocated at 0:15:55.240 --> 0:15:58.960 particular points on the Earth, and that's you're you're shifting 0:15:59.040 --> 0:16:01.720 from one set of just to another and the real 0:16:01.800 --> 0:16:04.640 genius of the program is how it allows you to 0:16:04.640 --> 0:16:06.880 to do that shift. Yeah, and it does in such 0:16:06.920 --> 0:16:09.160 a way where it kind of makes you feel like 0:16:09.240 --> 0:16:12.280 you are having a seamless experience, but in reality you 0:16:12.320 --> 0:16:15.160 are switching from one set of photos to another. There's 0:16:15.160 --> 0:16:18.440 a similar thing in a way, uh an idea called 0:16:18.600 --> 0:16:22.160 gigapan or gigapixel images. Gigapan is just one of the 0:16:22.240 --> 0:16:24.880 many terms for it. So this is the idea of 0:16:24.920 --> 0:16:28.920 taking several pictures high high, high resolution pictures of a 0:16:29.000 --> 0:16:32.280 scene and then stitching all those pictures together to kind 0:16:32.280 --> 0:16:36.200 of make sort of a panoramic image, but panoramic in 0:16:36.600 --> 0:16:38.960 beyond just you know, it's a very wide photo. It 0:16:39.000 --> 0:16:41.320 could be very tall. It could and and the cool 0:16:41.360 --> 0:16:44.480 thing is that allows you to zoom in at crazy 0:16:44.560 --> 0:16:47.800 levels because instead of it being just one big picture 0:16:47.800 --> 0:16:50.760 with lots and lots of information, it's actually a collection 0:16:50.800 --> 0:16:54.280 of mosaic of all these high resolution images. So I've 0:16:54.320 --> 0:16:57.360 seen some of like sporting events, like like the Olympics 0:16:57.440 --> 0:16:59.280 or something like that, and it's a picture of the 0:16:59.280 --> 0:17:02.080 crowd and when you first look at it, you just 0:17:02.120 --> 0:17:05.600 see a mass of faces. It's just a huge number 0:17:05.640 --> 0:17:08.679 of people, maybe a hundred thousand people, and then you 0:17:08.720 --> 0:17:11.160 could arbitrarily say, all right, don't want to zoom in 0:17:11.359 --> 0:17:13.720 on this one section of the crowd, and you zoom 0:17:13.720 --> 0:17:16.119 in until just that one section fills up your screen, 0:17:16.119 --> 0:17:18.719 and now you can suddenly see actual details, and then 0:17:18.760 --> 0:17:20.520 you say, they kind of want to zoom in on 0:17:20.560 --> 0:17:24.119 that collection of of of folks like that that small 0:17:24.240 --> 0:17:26.800 group of people right there, and zoom in even further, 0:17:27.200 --> 0:17:29.960 And depending upon how many photos they've taken and how 0:17:30.080 --> 0:17:31.879 high resolution the photos were, you might get to a 0:17:31.920 --> 0:17:34.399 point where you can read the text on a person's shirt, 0:17:35.000 --> 0:17:36.679 or at least be able to see what kind of 0:17:36.680 --> 0:17:39.360 basic design is on a person's clothing if they are 0:17:39.359 --> 0:17:41.440 wearing something that has a big logo on it or something, 0:17:41.480 --> 0:17:44.840 you might be able to tell um and it. The 0:17:44.840 --> 0:17:47.760 illusion is that you've got this one picture that you 0:17:47.800 --> 0:17:51.480 can just zoom in indefinitely, just like you could in 0:17:51.520 --> 0:17:54.760 the movies. But the way you produce that file is 0:17:54.800 --> 0:17:57.360 actually by taking all these different pictures. It's not like 0:17:57.440 --> 0:18:00.480 it is a single element the way you would think 0:18:00.560 --> 0:18:03.879 from a film or TV show. There's actually sort of 0:18:03.920 --> 0:18:08.200 a whole family of image processing techniques that are known 0:18:08.240 --> 0:18:12.880 as super resolution that's the idea of taking a picture 0:18:12.960 --> 0:18:17.600 and trying to somehow increase resolution after you've already got 0:18:17.680 --> 0:18:22.000 the final product. Um. One of the techniques that I 0:18:22.040 --> 0:18:25.359 think is interesting is, Uh, so we've talked about single 0:18:25.400 --> 0:18:31.199 frame increases in resolution, but what about multiple frames. So 0:18:31.359 --> 0:18:35.760 imagine you've got video and it's not my one image 0:18:35.760 --> 0:18:40.719 per second security camera, uh second per minute whatever I said. Um, 0:18:40.760 --> 0:18:46.080 it's it's like continuous video. You can actually put together 0:18:46.359 --> 0:18:52.800 frames inaggregate to make each frame sharper. Interesting. Actually, that's 0:18:52.800 --> 0:18:55.280 the way that the human eye works. That is basically 0:18:55.320 --> 0:18:57.760 how we are all seeing things all the time. We 0:18:57.920 --> 0:19:01.159 see in I mean, I guess you could call it 0:19:01.200 --> 0:19:03.919 still frames, but but basically in video, and uh, and 0:19:03.960 --> 0:19:06.640 our brains kind of compile the images as I'm looking 0:19:06.720 --> 0:19:09.240 back and forth between the two of you, or um 0:19:09.359 --> 0:19:11.280 or or kind of going like what's that weird thing 0:19:11.320 --> 0:19:14.520 on the corner behind old head? No, but um, now 0:19:15.160 --> 0:19:17.159 you know, and and and your brain puts together this 0:19:17.320 --> 0:19:21.120 information into more or less a single image. So so 0:19:21.320 --> 0:19:23.120 if you were to take a camera, let's say you've 0:19:23.119 --> 0:19:25.960 got a digital camera that could take burst photos like 0:19:25.960 --> 0:19:28.320 a whole bunch in just a blink of an eye 0:19:28.320 --> 0:19:30.679 that I assume you could apply the same sort of 0:19:30.720 --> 0:19:36.440 approach to try and create the best possible version of 0:19:36.480 --> 0:19:38.359 the picture you were trying to take. Oh, I'm sure. 0:19:38.400 --> 0:19:42.040 I mean what's operating here is that when you have 0:19:42.240 --> 0:19:46.560 multiple frames, each frame is probably giving you some type 0:19:46.600 --> 0:19:50.040 of information that wasn't available in the frame before. So 0:19:50.080 --> 0:19:53.040 if somebody's turning their head or something like that, at 0:19:53.080 --> 0:19:57.120 different points, you see different parts of it illuminated, um 0:19:57.240 --> 0:20:01.000 some parts are closer to the perfect ideal focus, and 0:20:01.000 --> 0:20:04.480 and so by sort of selecting the best part of 0:20:04.520 --> 0:20:08.280 each of those images and and averaging them right, you 0:20:08.280 --> 0:20:10.520 can get a sharper image than you had in any 0:20:10.560 --> 0:20:12.480 of the original frame. This is kind of like that 0:20:12.480 --> 0:20:15.679 that those commercials you see of the cameras where you 0:20:15.720 --> 0:20:20.119 can swap out people so that you just see the best, 0:20:20.840 --> 0:20:24.119 uh best faces for everyone. Like you've got the group photo, 0:20:24.200 --> 0:20:26.720 and you took a series of group photos, and you're like, well, 0:20:27.200 --> 0:20:30.159 little Billy was being a complete snot in the first 0:20:30.640 --> 0:20:32.400 for six of these, but in the seventh one he's 0:20:32.400 --> 0:20:36.480 looking he's looking, you know, at the camera and smiling. Unfortunately, 0:20:37.160 --> 0:20:40.040 Dad has his eyes half closed because he's just about 0:20:40.040 --> 0:20:43.440 the sneeze. So we need to combine all these photos 0:20:43.480 --> 0:20:47.320 into the ideal family photo that never existed, but seems 0:20:47.359 --> 0:20:50.600 to like that moment never existed, right, the moment where 0:20:50.720 --> 0:20:54.560 everyone in the family is smiling and content and behaving 0:20:54.720 --> 0:20:57.240 never existed, but you've created the illusion that it has 0:20:57.280 --> 0:21:00.280 by combining all these images into one. Or if you're 0:21:00.320 --> 0:21:02.480 George Lucas and you really like an actor's face in 0:21:02.520 --> 0:21:04.960 one take, but their body movements and another, so you 0:21:05.040 --> 0:21:08.119 just paste the two together in episodes one through three. Yeah, 0:21:08.280 --> 0:21:10.720 or you know, if you just don't like actors and 0:21:10.760 --> 0:21:15.560 tell them not to act. Sorry, that's a little editorializing there. 0:21:15.960 --> 0:21:20.000 This this also reminds me of the the app I 0:21:20.400 --> 0:21:23.159 talked to you about Lauren A Group GROUPI. Yeah, this 0:21:23.240 --> 0:21:26.160 is this is one um that was created by um 0:21:26.160 --> 0:21:29.440 I Do Use Labs, which is out in uh Pakistan. 0:21:29.840 --> 0:21:33.200 But it's um It's it's an app that let's everyone 0:21:33.280 --> 0:21:36.760 be in in a photograph without any one photographer having 0:21:36.800 --> 0:21:40.000 to having to step out or having to give your 0:21:40.040 --> 0:21:43.239 you know, very expensive device to a random strange may 0:21:43.320 --> 0:21:45.640