Speaker 1: Brought to you by Toyota. Let's go places. Welcome to
Speaker 1: Forward Thinking Either and Welcome to Forward Thinking, the podcast
Speaker 1: that looks at the future and says zoom manned enhance.
Speaker 1: I'm Jonathan Strickland, I'm Lauren Malcolm, and I'm Joe McCormick,
Speaker 1: and we wanted to talk about this idea of playing
Speaker 1: with cameras and the images that they take, whether it's
Speaker 1: still photography or video images, and this idea of being
Speaker 1: able to manipulate those images and maybe even do this
Speaker 1: crazy zoom and enhanced thing. You guys, You guys are
Speaker 1: familiar with the TV, right, I need some help you guys,
Speaker 1: what why is that there's been a theft? What was stolen? Well,
Speaker 1: somebody broke into my home and they stole all of
Speaker 1: my VHS tapes of the Super Mario Show. Wow, that's
Speaker 1: all of them. That's not necessarily a loss, but I
Speaker 1: appreciate that you're hurting Lauren. No, this is a loss
Speaker 1: not just for Joe over all mankind. It was the
Speaker 1: Super Mario Brothers Super Show and now I don't have it,
Speaker 1: but I've got a lead. Okay, So I had a
Speaker 1: digital security camera installed in my home, not too, as
Speaker 1: long as you have a VHS collection of Super Mario.
Speaker 1: But you have a digital security camera digital security camera,
Speaker 1: and it it records video in still frames of Oh,
Speaker 1: I guess they're about a hundred pixels each, so very
Speaker 1: low resolution. That's a hundred pixels, I mean. Well, anyway,
Speaker 1: so I got together puzzles that had a hundred pieces,
Speaker 1: so it's not that much. And it takes a picture
Speaker 1: every sixty seconds, so that's not bad. I don't call
Speaker 1: that video. I call that a series of very crappy
Speaker 1: still photos. But I have a photo of the thief, okay,
Speaker 1: all right, and I'm going to try to find him.
Speaker 1: It could also be sasquatch. We don't know you, but
Speaker 1: I need your help, okay, to identify this person. Right.
Speaker 1: So do we I know here that we got a
Speaker 1: lot of techie people here in the office. Do we
Speaker 1: have one of those enhanced computers? I can just scan
Speaker 1: this in, right, and and we can press enhance and
Speaker 1: it will give us the dudes face. No, we don't
Speaker 1: have one of those. Why don't we have They've got
Speaker 1: to be pretty cheap by now, right, No, So alright,
Speaker 1: so what what we're referring to? Here's this trope. I
Speaker 1: first of all, Joe, I certainly hope that that was
Speaker 1: just an example story that you haven't actually suffered a
Speaker 1: terrible loss of that. Just kidding, all right, good, So
Speaker 1: so the Super Mario videotapes are safe. Okay, good. So
Speaker 1: what we're talking about is a television trope. In fact,
Speaker 1: one of my favorite websites to just waste time on
Speaker 1: is one of the TV tropes sites where you can
Speaker 1: just read about all these these sort of cliches that
Speaker 1: have been used in television and film for about as
Speaker 1: long as the those types of media have existed. Really um,
Speaker 1: and some of them are a little newer than others.
Speaker 1: Zoom and enhanced, pretty new because we didn't have those
Speaker 1: kind of we didn't even think about those kind of
Speaker 1: capabilities until maybe the last couple of decades. Okay, so
Speaker 1: it may have been earlier than this. But the earliest
Speaker 1: example I can think of is in N two the
Speaker 1: movie Blade Runner Ridley Scott Harrison Ford um based on
Speaker 1: the Philip K. Dick. But there's a scene in it
Speaker 1: where Harrison Ford's character is doing some investigating. Yeah, he's
Speaker 1: got a photograph and he's got it on his computer
Speaker 1: screen and he keeps shuffling around, looking at different frames
Speaker 1: of the photograph and then zooming way in and enhancing
Speaker 1: the photo. Yeah, it's um, it's it's the way TV
Speaker 1: tropes defines it. They call it the enhanced button, very
Speaker 1: similar to the scenario you were talking about the very
Speaker 1: top of the show, Joe, He says, a staple of
Speaker 1: any crime drama. The enhanced button on the computer is
Speaker 1: able to turn a tiny, blurred, grainy image in a
Speaker 1: photo or video into a clear, unmistakable piece of evidence.
Speaker 1: This process is virtually instantaneous, unless added dramatic tension is required,
Speaker 1: in which case extra technobabble or more applied flabottanum may
Speaker 1: be needed. These, by the way, or other tropes may
Speaker 1: require someone to stand next to the computer intoning enhance,
Speaker 1: enhance for full effect. So yeah, this idea of taking
Speaker 1: something like an image a photograph that is imperfect, it
Speaker 1: has a limited amount of data in it, and then
Speaker 1: enhancing it so that it becomes something useful, something recognizable,
Speaker 1: is that a thing? Is that possible? All right? At
Speaker 1: least the way it's done in Hollywood? Not so much. Well,
Speaker 1: it's not possible the way they do it because in
Speaker 1: these movies and TV shows. You're just getting information that
Speaker 1: just blatantly just was not there before, right, Right, you
Speaker 1: can only enhance up to as much information as actually
Speaker 1: exists in the physical record of that infra exactly. You
Speaker 1: can't extrapolate new things whole cloth. Right The way, the
Speaker 1: way I'd put it is that there's sort of a
Speaker 1: bottleneck on data at the moment an image is captured.
Speaker 1: Um data comes into the camera and you record a
Speaker 1: certain amount of it, and then data comes out later. Now,
Speaker 1: you can manipulate that later data in all kinds of ways,
Speaker 1: but you can't ever put in more than came in
Speaker 1: through the lens to begin with, well, not not directly
Speaker 1: onto the raw image file. You can manipulate it enough
Speaker 1: and through guesswork. Really you're you're kind of you're kind
Speaker 1: of making some assumptions where you can fill in information
Speaker 1: that is missing. But it's not like you are uncovering
Speaker 1: the information that was there. You're actually creating new stuff
Speaker 1: to go in with the stuff that was captured at
Speaker 1: that moment. Right. It would be like if you saw,
Speaker 1: for example, somebody's shadow in a photograph. Right now, if
Speaker 1: somebody was a really good I don't know, this is
Speaker 1: a real thing, but it was really good at identifying humans.
Speaker 1: But the shadows they cast, they can do the same
Speaker 1: kind of thing. They can look at the shadow and
Speaker 1: the picture and say, oh, it's probably this person, But
Speaker 1: you still wouldn't actually have that person's image somewhere hidden
Speaker 1: in the file right. And you know, also, like if
Speaker 1: you've got a picture of the back of a person's head,
Speaker 1: it's not like the image of their face is contained
Speaker 1: in that data somewhere, and you can't just just rotate
Speaker 1: person and and have you know, there's no button to
Speaker 1: do that. It's amazing how often you can do that
Speaker 1: in science fiction though, like turn it around. I want
Speaker 1: to see this. I want to see this still image
Speaker 1: from the opposite angle, as if we can magically place
Speaker 1: the camera anywhere after the effect. Uh, what what you
Speaker 1: can do in a virtual environment? I mean, there are
Speaker 1: camera tricks you can do in a virtual environment that
Speaker 1: completely defy all laws of physics because that they aren't
Speaker 1: a problem in a virtual environment. So, for example, in
Speaker 1: a video game, there are a lot of video games
Speaker 1: where you can capture the footage of you playing while
Speaker 1: you're playing the video game, and then watch it later
Speaker 1: and you can even watch it from various camera angle
Speaker 1: is depending upon the type of game. Like, some games
Speaker 1: give you essentially free reign. You can place the camera
Speaker 1: anywhere you like. You can have a free roaming camera
Speaker 1: and move it dynamically as the scene plays back. So
Speaker 1: even though while you were playing you had one set
Speaker 1: of perceptions, you know, you might have been able to
Speaker 1: move the camera around then too, but you were limited
Speaker 1: at that time by whatever was happening right then and there.
Speaker 1: But in playback you may be given unlimited freedom. Now
Speaker 1: that's just not the case with real life. I mean,
Speaker 1: obviously you know it's that that that data exists in
Speaker 1: real life, but if there's nothing there to record it
Speaker 1: right then, So when I was talking about the you know,
Speaker 1: creating stuff so that you can fill in the gaps,
Speaker 1: it's where you're you're not again, you're not uncovering lost information.
Speaker 1: You're not or or obvious skated information. That's the way
Speaker 1: it comes across in these television and film examples where
Speaker 1: the answer is hidden in the file and you just
Speaker 1: have to dig it out. Yeah. Yeah, there's just a
Speaker 1: button on your computer that somehow makes it go from
Speaker 1: blurry to not blurry, and that it was the info
Speaker 1: was always there, it just needed to be not blurry.
Speaker 1: That's not the case. That's not the way it works. Um.
Speaker 1: There was a great article and Wired that talked about
Speaker 1: this and talked about some approaches that that various technicians
Speaker 1: have made to really uh address this issue. You know,
Speaker 1: I was talking about creating data to fill in those gaps,
Speaker 1: the stuff that's not actually there in these photos. You know,
Speaker 1: in the Hollywood and TV versions, it always seems like
Speaker 1: the information is there, it's just blurry or whatever. And
Speaker 1: when you hit this button, it removes whatever that problem
Speaker 1: was and you get to look at the information that
Speaker 1: was always there. That's not the case, right. We're talking
Speaker 1: about photos and video where stuff is missing. But you
Speaker 1: can start to fill in some of those gaps by
Speaker 1: creating stuff, by making guesses. And there was a great
Speaker 1: article and Wired that talked about this and talked about
Speaker 1: compressed sensing and sparsity. These are concepts that are used
Speaker 1: by technicians to kind of fill in information that might
Speaker 1: be missing, either a file. You know, maybe maybe the
Speaker 1: image is just really fuzzy. It could be an old photograph,
Speaker 1: or it could just have been made with a poor camera,
Speaker 1: or maybe something was on the lens or whatever, and
Speaker 1: we can use these techniques to try and fill in information. Now,
Speaker 1: in the Wired article, they had a really interesting analogy.
Speaker 1: They said that, imagine that you have a book where
Speaker 1: on one page of the book you have almost half
Speaker 1: or maybe even more of the information missing. So you've
Speaker 1: got just missing words in sentences. Now on the pages before,
Speaker 1: in the pages after, you've got some information there, But
Speaker 1: on the page that you're interested in your missing words.
Speaker 1: It would be like trying to extrapolate what those missing
Speaker 1: words were just based on the little bits of information
Speaker 1: you had on the previous and following pages. It's really
Speaker 1: difficult to do. However, with compressed sensing, there was an
Speaker 1: interesting development. There was a guy named Emmanuel Condez who
Speaker 1: was looking at what an image called the shep logan phantom. Now,
Speaker 1: this is actually an image that technicians you us in
Speaker 1: order to test imaging algorithms, and it kind of looks
Speaker 1: like a an alien with a slightly raised eyebrow, kind
Speaker 1: of a snarky alien. Now, what what he did was
Speaker 1: he took an older, fuzzy version of the image, not
Speaker 1: an older one, but a fuzzy one. So he was
Speaker 1: testing an algorithm, trying to see if a particular technique
Speaker 1: would allow him to uh to sharpen this image up
Speaker 1: a little bit. And the way the technique works is
Speaker 1: it looks for the simplest approach to filling out the
Speaker 1: information that's missing by look it's samples pixels in the image,
Speaker 1: and then from those samples, it starts to create simple
Speaker 1: shapes that are their color matched to the various parts
Speaker 1: of that sampled image. Now it tries to use the
Speaker 1: fewest number of these shapes to fill out this this
Speaker 1: photo or this this picture doesn't have to just be
Speaker 1: a picture. By the way, you can actually apply the
Speaker 1: same technique to other kinds of media, including music, where
Speaker 1: you know you might have a low sample rate for
Speaker 1: an old MP three and you want to try and
Speaker 1: enhance it. It could do the same sort of thing
Speaker 1: with that right right, Basically, anything with a with a
Speaker 1: wave form is going to operate under very similar principles.
Speaker 1: So what they're doing here is the reason why he
Speaker 1: was doing this in the first place was not just
Speaker 1: to get a sharper picture of an alien. The idea
Speaker 1: was to try and enhance m r I images because
Speaker 1: the best way to get a very very clear mr
Speaker 1: I image is to put someone in an MRI machine
Speaker 1: and have them stay perfectly still for a couple of minutes.
Speaker 1: But to stay perfectly still for a couple of minutes
Speaker 1: usually means having to put them under anesthesia. So they
Speaker 1: actually stopped breathing. That's how still they needed people to be. Now,
Speaker 1: that's not really that great an option. So he was
Speaker 1: looking at what if I took this approach to try
Speaker 1: and take an image that was captured in say forty
Speaker 1: seconds as opposed to two minutes, and then try to
Speaker 1: use this this uh, this simple technique to see if
Speaker 1: I can sharpen it up. So he runs the the
Speaker 1: image through this algorithm he's created and turned out that
Speaker 1: the resulting image was a perfect match two or a
Speaker 1: near perfect match to the UH the original version of
Speaker 1: the shep Logan phantom image. And he thought, well, that's weird.
Speaker 1: That can't be right. There's no way that worked. And
Speaker 1: so he tested it again and it got the same
Speaker 1: result and ended up showing it off to some other
Speaker 1: folks and and they really began to put their heads
Speaker 1: together and wrote a white paper on it and a
Speaker 1: research paper all about this UH technique, and um, yeah,
Speaker 1: it takes about a hundred thousand pixels for example, and
Speaker 1: just really focus on those and build out these shapes
Speaker 1: and it could build a usable image. But um, a
Speaker 1: couple of caveats. One is that it can take a
Speaker 1: few hours to do this as the algorithm goes through
Speaker 1: all the different variations of the simplest way of approaching this,
Speaker 1: And there is a chance that the resulting image that
Speaker 1: you get back at the end is not a match
Speaker 1: for what it should have been. There is that chance.
Speaker 1: It's a small chance according to the researchers, but it
Speaker 1: can happen because the computer is just basically guessing based
Speaker 1: upon all the little bit of information that it has.
Speaker 1: So in that sense, if you have this this area
Speaker 1: of doubt where you know, you might say, well, you know,
Speaker 1: according to our computer model, this is what the image
Speaker 1: would have looked like if we had looked at it
Speaker 1: at this scale. Uh, you you know, you have to
Speaker 1: keep that in mind. You have to remember, like, this
Speaker 1: is what it probably looks like, not this is definitely
Speaker 1: the image. So that's another difference from the Hollywood version.
Speaker 1: Right with the Hollywood one, as soon as they do
Speaker 1: the enhance, there's our guy, there's no way it's there's
Speaker 1: our guy. It's the guy who was built third in
Speaker 1: the film, so now we know we got him, whereas
Speaker 1: this is just some dude. Yeah, So I mean that's
Speaker 1: an interesting approach. And the whole idea of sparsity is
Speaker 1: this idea of going with the simplest um and fewest
Speaker 1: number of simple shapes. So it might be like it
Speaker 1: it detects a couple of blue pixels, uh in a
Speaker 1: in a or a few blue pixels in an area,
Speaker 1: and then it just fills out the rest of that
Speaker 1: area with the same color blue, so that it's approaching
Speaker 1: at you know, saying well this, this is probably part
Speaker 1: of a border for this thing. I'm just going to
Speaker 1: fill in the rest. And it does that thousands and
Speaker 1: thousands of times for the entire image. But yeah, it's
Speaker 1: still not the kind of instantaneous approach we see in
Speaker 1: popular films and television so well, on a much smaller
Speaker 1: scale and in that kind of instantaneous sort of time frame.
Speaker 1: That's that's a function of photoshop. I mean, you know
Speaker 1: you can you can click that I forget what the
Speaker 1: function is called in there. You can just click that
Speaker 1: little button and have it kind of fill out what
Speaker 1: a line would have looked like, yeah, yeah, there are
Speaker 1: too what's around it. There are a lot of algorithms
Speaker 1: out there that take this approach where it looks at
Speaker 1: the existing data and tries to extrapolate what the rest
Speaker 1: of it should be. Uh, and it's you know there
Speaker 1: and I'm talking about a few pixels, right, and they
Speaker 1: have different degrees of of uh of sophistication and and
Speaker 1: resolution and uh, but nothing on the scale of the
Speaker 1: Hollywood version. Sure, sure, I I do think that that
Speaker 1: what we see in those kind of everyday applications is
Speaker 1: is what leads to some of the confusion about what
Speaker 1: like the professionals can do. Um like a like like
Speaker 1: with Google Maps. You know, you can you can zoom
Speaker 1: and enhance in a Google map, but that's because it's
Speaker 1: built of these multiple tiles of images then, right, that
Speaker 1: that have greater acuity on the lower levels. It's not
Speaker 1: like when you are at you know, at like satellite
Speaker 1: view of the Earth that has all the same detail
Speaker 1: as it would if you are a low flying plane.
Speaker 1: They have different sets of images that are geolocated at
Speaker 1: particular points on the Earth, and that's you're you're shifting
Speaker 1: from one set of just to another and the real
Speaker 1: genius of the program is how it allows you to
Speaker 1: to do that shift. Yeah, and it does in such
Speaker 1: a way where it kind of makes you feel like
Speaker 1: you are having a seamless experience, but in reality you
Speaker 1: are switching from one set of photos to another. There's
Speaker 1: a similar thing in a way, uh an idea called
Speaker 1: gigapan or gigapixel images. Gigapan is just one of the
Speaker 1: many terms for it. So this is the idea of
Speaker 1: taking several pictures high high, high resolution pictures of a
Speaker 1: scene and then stitching all those pictures together to kind
Speaker 1: of make sort of a panoramic image, but panoramic in
Speaker 1: beyond just you know, it's a very wide photo. It
Speaker 1: could be very tall. It could and and the cool
Speaker 1: thing is that allows you to zoom in at crazy
Speaker 1: levels because instead of it being just one big picture
Speaker 1: with lots and lots of information, it's actually a collection
Speaker 1: of mosaic of all these high resolution images. So I've
Speaker 1: seen some of like sporting events, like like the Olympics
Speaker 1: or something like that, and it's a picture of the
Speaker 1: crowd and when you first look at it, you just
Speaker 1: see a mass of faces. It's just a huge number
Speaker 1: of people, maybe a hundred thousand people, and then you
Speaker 1: could arbitrarily say, all right, don't want to zoom in
Speaker 1: on this one section of the crowd, and you zoom
Speaker 1: in until just that one section fills up your screen,
Speaker 1: and now you can suddenly see actual details, and then
Speaker 1: you say, they kind of want to zoom in on
Speaker 1: that collection of of of folks like that that small
Speaker 1: group of people right there, and zoom in even further,
Speaker 1: And depending upon how many photos they've taken and how
Speaker 1: high resolution the photos were, you might get to a
Speaker 1: point where you can read the text on a person's shirt,
Speaker 1: or at least be able to see what kind of
Speaker 1: basic design is on a person's clothing if they are
Speaker 1: wearing something that has a big logo on it or something,
Speaker 1: you might be able to tell um and it. The
Speaker 1: illusion is that you've got this one picture that you
Speaker 1: can just zoom in indefinitely, just like you could in
Speaker 1: the movies. But the way you produce that file is
Speaker 1: actually by taking all these different pictures. It's not like
Speaker 1: it is a single element the way you would think
Speaker 1: from a film or TV show. There's actually sort of
Speaker 1: a whole family of image processing techniques that are known
Speaker 1: as super resolution that's the idea of taking a picture
Speaker 1: and trying to somehow increase resolution after you've already got
Speaker 1: the final product. Um. One of the techniques that I
Speaker 1: think is interesting is, Uh, so we've talked about single
Speaker 1: frame increases in resolution, but what about multiple frames. So
Speaker 1: imagine you've got video and it's not my one image
Speaker 1: per second security camera, uh second per minute whatever I said. Um,
Speaker 1: it's it's like continuous video. You can actually put together
Speaker 1: frames inaggregate to make each frame sharper. Interesting. Actually, that's
Speaker 1: the way that the human eye works. That is basically
Speaker 1: how we are all seeing things all the time. We
Speaker 1: see in I mean, I guess you could call it
Speaker 1: still frames, but but basically in video, and uh, and
Speaker 1: our brains kind of compile the images as I'm looking
Speaker 1: back and forth between the two of you, or um
Speaker 1: or or kind of going like what's that weird thing
Speaker 1: on the corner behind old head? No, but um, now
Speaker 1: you know, and and and your brain puts together this
Speaker 1: information into more or less a single image. So so
Speaker 1: if you were to take a camera, let's say you've
Speaker 1: got a digital camera that could take burst photos like
Speaker 1: a whole bunch in just a blink of an eye
Speaker 1: that I assume you could apply the same sort of
Speaker 1: approach to try and create the best possible version of
Speaker 1: the picture you were trying to take. Oh, I'm sure.
Speaker 1: I mean what's operating here is that when you have
Speaker 1: multiple frames, each frame is probably giving you some type
Speaker 1: of information that wasn't available in the frame before. So
Speaker 1: if somebody's turning their head or something like that, at
Speaker 1: different points, you see different parts of it illuminated, um
Speaker 1: some parts are closer to the perfect ideal focus, and
Speaker 1: and so by sort of selecting the best part of
Speaker 1: each of those images and and averaging them right, you
Speaker 1: can get a sharper image than you had in any
Speaker 1: of the original frame. This is kind of like that
Speaker 1: that those commercials you see of the cameras where you
Speaker 1: can swap out people so that you just see the best,
Speaker 1: uh best faces for everyone. Like you've got the group photo,
Speaker 1: and you took a series of group photos, and you're like, well,
Speaker 1: little Billy was being a complete snot in the first
Speaker 1: for six of these, but in the seventh one he's
Speaker 1: looking he's looking, you know, at the camera and smiling. Unfortunately,
Speaker 1: Dad has his eyes half closed because he's just about
Speaker 1: the sneeze. So we need to combine all these photos
Speaker 1: into the ideal family photo that never existed, but seems
Speaker 1: to like that moment never existed, right, the moment where
Speaker 1: everyone in the family is smiling and content and behaving
Speaker 1: never existed, but you've created the illusion that it has
Speaker 1: by combining all these images into one. Or if you're
Speaker 1: George Lucas and you really like an actor's face in
Speaker 1: one take, but their body movements and another, so you
Speaker 1: just paste the two together in episodes one through three. Yeah,
Speaker 1: or you know, if you just don't like actors and
Speaker 1: tell them not to act. Sorry, that's a little editorializing there.
Speaker 1: This this also reminds me of the the app I
Speaker 1: talked to you about Lauren A Group GROUPI. Yeah, this
Speaker 1: is this is one um that was created by um
Speaker 1: I Do Use Labs, which is out in uh Pakistan.
Speaker 1: But it's um It's it's an app that let's everyone
Speaker 1: be in in a photograph without any one photographer having
Speaker 1: to having to step out or having to give your
Speaker 1: you know, very expensive device to a random strange may
Speaker 1: may not make off with. So let's say let's say
Speaker 1: like we get the whole House Stuff Works crew to
Speaker 1: go someplace. You know, we're all going to six Flags
Speaker 1: for a day, and we want to get our our
Speaker 1: picture taken in front of the great American screen machine
Speaker 1: as we are wont to do. And uh, and there's
Speaker 1: the whole group. But who takes the photo? Do we
Speaker 1: and trust our expensive house stuff works camera to some
Speaker 1: ragamuffin walking by or bugs bunny? Yeah, he can't be trusted.
Speaker 1: He can't doesn't even have opposable thumbs. So yeah, we
Speaker 1: we end up saying, well, what can we do? What
Speaker 1: if we use group pick? Then essentially, from what I understand,
Speaker 1: what allows you to do is take at least two
Speaker 1: photos where you swap out photographers, and then you can
Speaker 1: combine the two so that you have both photographers in
Speaker 1: the full group photo. All Right, the app app helps
Speaker 1: you frame a picture and then um you uh, you know,
Speaker 1: one person takes the first picture, a second person takes
Speaker 1: the second one. Um, you mark out who the two
Speaker 1: photographers were on screen, and uh, based on the fact
Speaker 1: that it's already framed it for you and so they're
Speaker 1: more or less identical photos. Otherwise, um, it will it
Speaker 1: will swap out the little slivers well, and you know
Speaker 1: it's it helps if you have one each each photographer
Speaker 1: on the extreme ends of the frame. Right, this would
Speaker 1: have been so useful to like, uh, despotic Soviet rulers,
Speaker 1: you know, like Joseph Stalin, But you went straight to
Speaker 1: despotic Soviet rulers. Yeah, you know. So, so you've eliminated
Speaker 1: some political rival and you want to erase his image
Speaker 1: from pictures of you. Um now you cannot just erase him,
Speaker 1: but you can also insert your new cronies and sycophants. Right.
Speaker 1: I said that what I want to do is use
Speaker 1: this kind of thing to take a picture where there's
Speaker 1: like twenty Jonathan's all in the same photo. You can
Speaker 1: you can certainly do that your dream world, you know,
Speaker 1: but my dream, your nightmare, this all this all weirds
Speaker 1: me out, honestly. I I mean, it's the technology is
Speaker 1: fascinating and wonderful. Um and and this kind of automatically
Speaker 1: revisionist history is I'm not sure that whether or not
Speaker 1: I need access to this technology for for a nap.
Speaker 1: It's it actually does have some somewhat troubling implication. The
Speaker 1: idea that you can manipulate images to such an extent
Speaker 1: as to create a new history that never really existed.
Speaker 1: It's kind of you know, I mean, that's a plot
Speaker 1: point and a lot of movies and television as well.
Speaker 1: It's just now we're getting to a point where the
Speaker 1: average consumer could theoretically do that with very little training,
Speaker 1: and and we all do this all the time. I
Speaker 1: mean everything that we're seeing again, like the human eye
Speaker 1: is flawed. It's only taking in so much information and
Speaker 1: it's filling in a lot of gaps in between those
Speaker 1: frames that it's taking in. But um, but yeah, just
Speaker 1: doing that on purpose. I'm like, Okay, well, I thought
Speaker 1: i'd talk a little bit about some other kind of
Speaker 1: cool camera tricks. There was one in particular one to
Speaker 1: talk about, um, which was this idea of being able
Speaker 1: to take photos and then change the focal point after
Speaker 1: you've taken the photo. Yeah. The light field cameras. Yeah,
Speaker 1: lightfield cameras also known as plan optic cameras, although they're
Speaker 1: not true plan optic cameras. A plan optic camera, well,
Speaker 1: it comes from the word plan us, which actually means
Speaker 1: full or complete, and then optic, of course, is the
Speaker 1: behavior of light. A true plan optic camera is impossible.
Speaker 1: It's just a theory. It's kind of a thought experiment,
Speaker 1: because the reason why it's impossible is that it's the
Speaker 1: idea that you would be able to take, uh, all
Speaker 1: the visual information within an environment, kind of like in
Speaker 1: those virtual environments, and be able to reproduce a still
Speaker 1: image from any angle, from any focal point. It's not
Speaker 1: really possible, not only because we can't just place a
Speaker 1: camera anywhere in the room, but also because the camera
Speaker 1: itself is going to reflect light off of it, and
Speaker 1: so the camera's presence. It's kind of like that whole idea,
Speaker 1: like the by observing something, you change the observed a
Speaker 1: sort of camera similar although Heisenberg's uncertainty principle would tell
Speaker 1: me that I know where the camera is, but I
Speaker 1: don't know how quickly it's taking pictures. Um, that's just
Speaker 1: a little quantum joke. But anyway, it's not a true
Speaker 1: plan optic camera, but light field camera. What does is
Speaker 1: it tries to capture all the rays of light and
Speaker 1: every direction that they are traveling within a single frame
Speaker 1: of reference, a single image. So the camera that a
Speaker 1: lot of people have heard about is the light tro
Speaker 1: which is this, uh, this really cool camera. If you
Speaker 1: were to just look at one, you would you would
Speaker 1: think that looks like some sort of prism or something,
Speaker 1: because it's not, you know, it's not camera shaped. It
Speaker 1: looks like you know, this this elongated uh cubic kind
Speaker 1: of thing, and it actually allows you to take photos
Speaker 1: and then change the focal point after you've taken them.
Speaker 1: So if you've set up like a scene so that
Speaker 1: you've got you know, Joe, let's say that you are
Speaker 1: just crazy about war gaming, and you haven't you have
Speaker 1: an enormous collection of painted lead miniatures, oh that kind
Speaker 1: of war gaming. No, not that you actually are buddying
Speaker 1: up to your you know, Russian despotic friends that you've
Speaker 1: already referred to in this episode, But no, that you
Speaker 1: play a tabletop war gaming games and you've got a
Speaker 1: huge collection of these painted lead miniatures and you've you've
Speaker 1: set them up into this neat uh row upon row,
Speaker 1: a battalion of soldiers and on this table, and you
Speaker 1: take a photo from the end of the table. Now,
Speaker 1: normally you would have to set a focal point on
Speaker 1: your camera, right, you would have to say, all right,
Speaker 1: I want to focus on the front soldiers, so that
Speaker 1: everything in the background kind of fades away into fuzziness.
Speaker 1: As it goes further back, or you would set the
Speaker 1: focal point so that the ones in the back are
Speaker 1: in focus and the ones in front are kind of blurry. Well,
Speaker 1: the litro camera captures the light field, all of those
Speaker 1: light rays traveling in every direction, and then creates essentially
Speaker 1: a virtual camera with a virtual lens within the software.
Speaker 1: And so when you view your image, you can change
Speaker 1: the focal point and say, all right, I want to
Speaker 1: focus on the soldiers that are in the back, and
Speaker 1: it'll switch the focus to the soldiers in the back,
Speaker 1: or I want to focus on the ones in the front,
Speaker 1: and it will create essentially a virtual camera with a
Speaker 1: virtual lens and a virtual image sensor that it have
Speaker 1: created that particular image, and you can change as many
Speaker 1: times as you want. Uh. And it lives that way,
Speaker 1: but only if you're viewing it on a computer. Obviously,
Speaker 1: if you were to ever print an image out, it
Speaker 1: would be stuck in whatever focal point you had chosen.
Speaker 1: That seems really cool, but I wonder how much space
Speaker 1: does one of those image files take up? A lot? Yeah,
Speaker 1: they does it take a long time to to process that?
Speaker 1: Not at all? It's like crazy fast. I mean, does
Speaker 1: it take a long time to take the original image.
Speaker 1: Not at all. I mean, because you know the kind
Speaker 1: of depth of field the ear that you're talking about, Like,
Speaker 1: you know, what was so revolutionary about artists like Ansel
Speaker 1: Adams was that they were working with such large prints
Speaker 1: of film that they could gain a depth of focus
Speaker 1: that was huge. Well, it is. There is a limitation
Speaker 1: the LTRO camera. It's not like it's not the kind
Speaker 1: of camera you're gonna take with you to go um,
Speaker 1: you know, like on a fast sightseeing tour. It's it's
Speaker 1: great for things that where you have composed a scene
Speaker 1: and you want to take a photo of that scene,
Speaker 1: or if you wanted to do something like you have
Speaker 1: a flower in front of you, and you're holding a
Speaker 1: flower in front of you and there's the Eiffel Tower
Speaker 1: in the background, you could take a picture like that
Speaker 1: and then you could swap the focus so that the
Speaker 1: Eiffel towers and focus are the flowers and focus. But
Speaker 1: it's not the kind of thing you would just carry
Speaker 1: around to take a snap whenever you were walking around.
Speaker 1: It's it's not that that kind of cameras. It's not
Speaker 1: your no, no, it's not designed for that. It's not
Speaker 1: meant for that. UH. And you know I've I've played
Speaker 1: with one of these. I actually got a chance to
Speaker 1: play with one, and it's kind of cool. The viewfinder
Speaker 1: on the back is essentially the entire uh. It's not
Speaker 1: even if you find her. It's a screen. It's a
Speaker 1: touch screen that is the entire uh interface. So you'd
Speaker 1: point the camera at something, you'd see the video image
Speaker 1: of it on the little of the screen on the back.
Speaker 1: It keeps saying viewfinder, but it's a screen. You tap it,
Speaker 1: it would take the photo, uh, and then you could
Speaker 1: look at the image on the screen on the back,
Speaker 1: and even in you could touch different parts of the
Speaker 1: image and bring that part into focus. UH. And then
Speaker 1: once you upload the images, they would live on a
Speaker 1: website that LTRO owned, and you would be able to
Speaker 1: play with them and share them. That way, you could
Speaker 1: share it onto other platforms like Facebook or Twitter or whatever.
Speaker 1: People could view the images and and they could change
Speaker 1: the focal point too. So it's a living image in
Speaker 1: that sense. So if I were to upload one of
Speaker 1: these images, Joe, you could go and look at and say, oh,
Speaker 1: what would this look like if this part we're in
Speaker 1: focus and you click on it and then it would switch,
Speaker 1: so it was a new zoom in and see the
Speaker 1: shadowy man in the window. That part is not built
Speaker 1: in yet, but who knows what could happen to the future.
Speaker 1: So that was one super cool kind of futuristic thing
Speaker 1: that exists right now. And the lightro camera came out
Speaker 1: a couple of years ago, and has you know, sort
Speaker 1: of been uh more or less a curiosity, I would
Speaker 1: say among a certain like like the tech savvy group
Speaker 1: who heard about it early. Uh, kind, I've really dug it.
Speaker 1: I don't own one. I thought it was a neat product,
Speaker 1: but I didn't actually purchase one, but I didn't enjoy
Speaker 1: playing with it. But then there's other like kind of
Speaker 1: futuristic ideas, like the idea of being able to use
Speaker 1: a camera to take a picture of something that's not
Speaker 1: even in the room, like it's in the room next door.
Speaker 1: How does that work? Well, you could have an X
Speaker 1: ray camera that would work, but it would also be
Speaker 1: very dangerous. Are radiating yourself every time you take a picture? Yeah,
Speaker 1: not to mention, not to mention your your don't do
Speaker 1: it to yourself, you do it to other people. But
Speaker 1: if you're taking the photo you're still being exposed to
Speaker 1: X ray. Get my lead pants. We're doing photography, uh
Speaker 1: lead pants at six Flags. That's that's what I'm picking
Speaker 1: up from this. But anyway, so this is a concept
Speaker 1: that has been worked out over at M I T.
Speaker 1: Settle Children Settle, they called him lead pas. Okay, Joe,
Speaker 1: enough enough, we're keeping all of this. It's all going in. Yes,
Speaker 1: we are executive decision. It's all kept. Noel, you answer
Speaker 1: to me. It's all kept. So um M I T
Speaker 1: and M I T researchers were working on this idea
Speaker 1: of being able to take images of stuff that wasn't
Speaker 1: directly within the field of view of the camera. The
Speaker 1: example used was that let's say you're shooting an image
Speaker 1: in a room and there's an open door that goes
Speaker 1: into another room. Now you do not have an angle
Speaker 1: of you into that other room. You can just see
Speaker 1: the open door that has opened into the other room.
Speaker 1: You take an image with this camera, and then it
Speaker 1: starts to collect data and reconstruct what might be in
Speaker 1: that other room, giving you an image of let's say
Speaker 1: that there's a person hiding in there. You would see
Speaker 1: the picture of a figure in that other room, which
Speaker 1: is a cool idea. How does this work. It's actually
Speaker 1: using very very short bursts of laser light to project
Speaker 1: laser light out that some of that laser light hits
Speaker 1: the doorway that's open and bounces off of it, and
Speaker 1: then will eventually hit stuff that's in the room, bounce
Speaker 1: off that back to the door, bounce off the door
Speaker 1: back to the camera. Now, the number of rays of
Speaker 1: light of this laser light, or the the amount of
Speaker 1: information that's coming back is a fraction of what was
Speaker 1: sent out. Right, You're you're only getting a tiny little
Speaker 1: echo back of what you had just sent out in
Speaker 1: a burst. And they're using femto lasers, which means it's
Speaker 1: sending out a burst of of light that's a quadrillionth
Speaker 1: of a second long. And uh, they actually have to
Speaker 1: use a special kind of shutter that closes after they
Speaker 1: shoot out this light because they don't want that initial
Speaker 1: bounce back to affect the information of the stuff that's
Speaker 1: in that of the room because anything that if you know,
Speaker 1: if if the laser light just hits the door and
Speaker 1: bounces right back to the camera, that's going to ruin
Speaker 1: the image. So what does is it The shutter actually
Speaker 1: stays closed for a fraction of a second, then opens
Speaker 1: up to accept all more all incoming photons, and then
Speaker 1: the way it reconstructs the room that is out of
Speaker 1: you is it measures the amount of time it took
Speaker 1: that photon to come back to the camera. So it's
Speaker 1: almost like sonar, but with light um and it's a
Speaker 1: really cool idea. The only thing is that the reconstruction part,
Speaker 1: again is probabilistic. It's the best guess, which means that
Speaker 1: you could get information back because it's such a small
Speaker 1: amount compared to what you sent out that there's a
Speaker 1: lot of extrapolation that has to happen in order for
Speaker 1: you to be able to take a look at what
Speaker 1: was in that other room. Yeah, i'd wonder if your
Speaker 1: image of what was in the room could be affected
Speaker 1: by I don't know, I mean not just solid objects,
Speaker 1: but heat and atmospheric composition. I would imagine not I mean,
Speaker 1: we're talking lasers, so that's a very direct kind of
Speaker 1: it's measuring them the time periods between the photons. Then
Speaker 1: it's uh, yeah, I think I don't I mean, I
Speaker 1: honestly don't know the answer to that. It may very
Speaker 1: well be um, I think most of the time, like
Speaker 1: the the at least the example they gave of taking
Speaker 1: a picture of something that's happening in another room. You
Speaker 1: probably don't have to worry too much about that. Now.
Speaker 1: If there's a lot of electromagnetic interference in there, that
Speaker 1: could end up playing a part. But I don't know.
Speaker 1: Maybe if electro is is trying to play Xbox in
Speaker 1: the room next door, that might be a bad thing.
Speaker 1: Extra technology combined in with them with with regular old photography,
Speaker 1: if you can call something like that regular old photography,
Speaker 1: is um is probably gonna going to lead us in
Speaker 1: interesting places. Like you know, if all of our all
Speaker 1: of our cell phones basically have accelerometers in them these days,
Speaker 1: and if you can combine that data with the data
Speaker 1: that happened when you took a motion blurred photo, you
Speaker 1: could hypothetically correct for so yeah, so what essentially saying like, oh, well,
Speaker 1: the camera was moving right to left when this photo
Speaker 1: was taken. Uh, here is what it would have looked
Speaker 1: like had it been still at the moment that the
Speaker 1: photo was snapped. Interesting. Yeah, the cool thing about this
Speaker 1: technology is that while we are not in the realm
Speaker 1: of zoom and enhanced the way we see it in
Speaker 1: movies and television. There's no question in my mind that
Speaker 1: we are heading in that direction now. It may very
Speaker 1: well be that the images we see when we zoom
Speaker 1: and enhance are a lot of guesswork, very sophisticated guesswork.
Speaker 1: But I think we're going to get there to a
Speaker 1: point where we don't have to wait a certain number
Speaker 1: of ten hours to to get an idea of what
Speaker 1: this fuzzy photo might be of. Well, in a way,
Speaker 1: I would say that I do think in one interpretation,
Speaker 1: we are never going to get there because we're still
Speaker 1: never going to have information that wasn't no, no, we
Speaker 1: will always be limited to what's there and guesses about it. Right,
Speaker 1: But but our guesses are getting better, and our way
Speaker 1: of recording is getting better. So like we previous cameras
Speaker 1: did not have this shoot a femto laser in new
Speaker 1: a you know, obscured room, right, And as computers get
Speaker 1: more powerful, we're um, you know a lot of the
Speaker 1: equations that people are working with right now are things
Speaker 1: that have been around since the nineteen hundreds or the
Speaker 1: eighteen hundreds. I'm sorry, um, you know, the the Furrier transform,
Speaker 1: which is a big one that um that is being
Speaker 1: passed around in most of the most of the apps
Speaker 1: that you can download to reduce blur. Yeah, that was
Speaker 1: a dude who was born in seventy eight. So so
Speaker 1: you know that this this math has been around, but
Speaker 1: the way that we're using it right now it's pretty
Speaker 1: pretty incredible. Yeah. So, uh, we're not going to be
Speaker 1: we're not going to be doing any sort of Bones
Speaker 1: like technological wonders. I hear that they actually started to
Speaker 1: scale back on some of the more ridiculous technological things
Speaker 1: they would do in that show. I haven't watched it.
Speaker 1: Watch it in any seasons? Yeah, So which one is Bones?
Speaker 1: Bones was the one with um David Boreennas as the
Speaker 1: cop and uh Emily Emily Deschanelle as a as the
Speaker 1: anthropologist and um and they and they had their there
Speaker 1: they solve crimes through science, science and quotes science fiction.
Speaker 1: They're they're pretty lady. Um computer scientist would would would
Speaker 1: be like like, oh hey, yeah, no, I just totally
Speaker 1: developed this new computer that does this crazy things. So yeah,
Speaker 1: I'm excited to see where camera technology takes us in
Speaker 1: the future. Maybe we get to a point where every
Speaker 1: image you see you'll just have to keep in mind,
Speaker 1: I cannot trust that this moment ever actually happened. We're
Speaker 1: kind of already there, aren't we, Because I don't know.
Speaker 1: Do you believe that photos you see on the internet,
Speaker 1: Like when my friends on Facebook post their wedding photos
Speaker 1: is a picture of their new baby. I have to
Speaker 1: comment shopped. This has been photoshop? Did you guys see this? Is?
Speaker 1: This plays into our conversation a little bit. Did you
Speaker 1: guys see the thing I posted? There was a guy
Speaker 1: who faked being at Comic Con. Yeah, Like a friend
Speaker 1: of his went to Comic Con and he was not
Speaker 1: going to Comic Con, but he decided, just for kicks
Speaker 1: that he would pretend that he was also at Comic Con.
Speaker 1: So he kept texting him going like, oh, hey, I'm
Speaker 1: over here in this meeting room, or are are you here?
Speaker 1: Oh you just missed me. Oh no, I'm over here now.
Speaker 1: But he was. He had been before, so he knew
Speaker 1: enough about Comic Con to be able to, uh to
Speaker 1: to fake it and say, oh, I'm over at hall
Speaker 1: h or I'm over across at the over in the
Speaker 1: gas Lamp district getting food and just leading this poor
Speaker 1: sucker along the entire time. One of the things he
Speaker 1: did was he found he scoured Twitter for images taken
Speaker 1: an instagram taken from from comic Con, found one of
Speaker 1: these two guys, like it was a couple of celebrities
Speaker 1: posing together, and so then he matched himself, uh, standing
Speaker 1: in an alley way behind his house, and then photoshopped
Speaker 1: himself into the photo, placing the guy one of the
Speaker 1: two celebrities, so it looked like he was hanging with
Speaker 1: one of these other guys, and then uploaded that and
Speaker 1: sent it to his friends, say, I just ran into
Speaker 1: so and so here we are together, and it looked great.
Speaker 1: I mean it didn't like upon casual glance, it did
Speaker 1: not appear to be a photoshoped image. Now, if you
Speaker 1: were to look really closely, you'd say the lighting is
Speaker 1: really weird because his face has not lit exactly the
Speaker 1: same way. But you know, if you're just looking casually,
Speaker 1: you wouldn't think anything of it. So Yeah, at this point,
Speaker 1: I think you're right, Joe. I think we have to
Speaker 1: just assume that everyone's photos of their wedding and babies
Speaker 1: and everything is chopped. Yeah, everything is well, guys. Uh,
Speaker 1: that kind of wraps up our discussion. What do you
Speaker 1: guys think about the future of photography and videography and
Speaker 1: cameras in general. Is it's something that's exciting to you.
Speaker 1: Are you a photoshop wizard? Do you have lots of
Speaker 1: examples of crazy photoshops? Do you want to do some
Speaker 1: crazy photoshops of the hosts of Forward Thinking. It's gonna happen.
Speaker 1: I might as well ask it happens. Well, guess what.
Speaker 1: We have images of all the hosts of Forward Thinking.
Speaker 1: You can find those over at our Facebook page. They're
Speaker 1: they're up there. You can also if you hunt around,
Speaker 1: you'll you'll find photos of us. I can't wait to
Speaker 1: see where we end up. By the way, if you
Speaker 1: want to get in touch with us, you can email
Speaker 1: us our addresses fw thinking at discovery dot com and
Speaker 1: go to fow thinking dot com for all the blogs, podcasts, videos,
Speaker 1: lots of other interesting material there. We look forward to
Speaker 1: hearing from you, and we'll talk to you again really soon.
Speaker 1: For more on this topic in the future of technology,
Speaker 1: visit forward thinking dot com. Brought to you by Toyota.
Speaker 1: Let's go places