Speaker 1: Welcome to tech Stuff, a production from I Heart Radio.
Speaker 1: Hey there, and welcome to tech Stuff. I'm your host,
Speaker 1: Jonathan Strickland. I'm an executive producer with I Heart Radio.
Speaker 1: And how the tech are you? I got a request
Speaker 1: on Twitter from Bronson Adams to talk about the tech
Speaker 1: of the volume, which is extensively used in the more
Speaker 1: recent Star Wars projects that we have seen, particularly on
Speaker 1: Disney Plus, as well as the A R Wall which
Speaker 1: is being used for Star Trek projects. So once again
Speaker 1: it's Star Wars versus Star Trek. These two technologies are
Speaker 1: actually quite similar to one another, so I figure we
Speaker 1: can tackle both of them together, and right at the top,
Speaker 1: I'm going to say that these technologies in some ways
Speaker 1: actually harkened back to ye olden days of filmmaking, and
Speaker 1: in other ways they are an attempt to address the
Speaker 1: problem ms that come with setting a film in a
Speaker 1: fantastical place, and that includes, you know, worlds that just
Speaker 1: don't exist. So how do you bring those to life
Speaker 1: in a way that's believable and immersive so that you know,
Speaker 1: you're actors don't have any issues performing within the space,
Speaker 1: and your audience hopefully totally buys into it. So when
Speaker 1: you're doing that second thing, that is when you are
Speaker 1: trying to create a world that doesn't really exist and
Speaker 1: make it exist on film, you're putting a lot of
Speaker 1: pressure on yourself and the actors. Maybe you have a
Speaker 1: massive budget and you can actually build sets so that
Speaker 1: your fantastical world is more or less realized, but that
Speaker 1: might not be a practical solution, and maybe instead you
Speaker 1: film everything against a green screen, or maybe you have
Speaker 1: an entire set where there are elements that are greened
Speaker 1: out and then you use the chroma key process to
Speaker 1: replace it with, you know, whatever is supposed to be there.
Speaker 1: If you've ever seen shots of the prequel trilogy from
Speaker 1: Star Wars when it was in production, you'll see like
Speaker 1: there were entire sets where they were just like arches
Speaker 1: and things that were all decked out in green to
Speaker 1: be replaced later. But then imagine the actors who are
Speaker 1: in that place, who are trying to perform in what
Speaker 1: is essentially a featureless green environment and still do a
Speaker 1: believable job that is not easy. If you need any
Speaker 1: evidence about how challenging this can be, even for revered actors,
Speaker 1: you can listen to Sir Ian McKellen talk about it
Speaker 1: in the commentary for The Hobbit and Unexpected Journey, though
Speaker 1: I would understand it if you didn't want to do that,
Speaker 1: because it would mean having to watch The Hobbit. Now.
Speaker 1: I say that as a dude who hasn't had to
Speaker 1: en Elvish on my left arm. I'm a huge Hobbit fan.
Speaker 1: It's my favorite book in the world. I am not
Speaker 1: a fan of the films anyway. Mckillan said that he
Speaker 1: was absolutely miserable acting in these green screen environments against
Speaker 1: no one at all, because you know, all of the
Speaker 1: elements of the scene that he was supposed to be
Speaker 1: reacting to they would all be added in later. And
Speaker 1: he said that he actually considered quitting acting if that
Speaker 1: is what acting now was going to be, which is
Speaker 1: a big old roof. But the volume and the A
Speaker 1: R wall both are systems that can project stuff that
Speaker 1: otherwise wouldn't be within the scene on L E. D screens.
Speaker 1: Now that means the actors are not just sitting there
Speaker 1: trying to imagine what's supposed to be around them and
Speaker 1: hoping that they get it right. You don't have to
Speaker 1: worry about five different actors having five different eye lines
Speaker 1: towards some imaginary spot because they can actually see the
Speaker 1: thing that they're supposed to be looking at, it could
Speaker 1: be much more immersive with some restrictions. Now, before I
Speaker 1: get into all that, I think it would behoove us
Speaker 1: to talk about some of the methods that came before
Speaker 1: where filmmakers were trying to get around these limitations of reality.
Speaker 1: Filmmaking has always been about finding ways to push against
Speaker 1: those limited aations, and some of those efforts worked to
Speaker 1: a degree, and some, especially in the early days, didn't.
Speaker 1: So let's talk about some very early visual effects or
Speaker 1: VFX techniques that relate to one another. And a lot
Speaker 1: of this material is stuff that I have talked about
Speaker 1: in previous episodes, so I'm not going to go into
Speaker 1: exhaustive detail, but it really falls under this general topic
Speaker 1: of compositing. Now. In film, compositing is where you're doing
Speaker 1: the incorporation of two or more images from separate sources
Speaker 1: into a final shot. For example, a typical green screen
Speaker 1: application would see a character in the foreground that would
Speaker 1: be an image from source number one. Right, you had
Speaker 1: your primary camera on your main character against a green background.
Speaker 1: That background would be coming from a second image source
Speaker 1: so that would be image number two. Maybe it's a
Speaker 1: filmed source. Maybe you went out and filmed on locations inplace,
Speaker 1: and you know it wasn't practical to get the actor
Speaker 1: out to that location, or maybe it wasn't safe, and
Speaker 1: so you pair them that way. Maybe the background's computer generated,
Speaker 1: maybe it never existed in the first place, doesn't really matter.
Speaker 1: The point is the finished scene, the finished film is
Speaker 1: this composite of these two images into a final image
Speaker 1: to make the illusion that that character was in that
Speaker 1: location at that time. So your foreground characters appearing in
Speaker 1: front of wherever you wanted them to be. Now, way
Speaker 1: back at the turn of the twentieth centuries, around nineteen
Speaker 1: a filmmaker named George Mellier was pioneering all kinds of techniques.
Speaker 1: Like you want to talk about a a brilliant filmmaker.
Speaker 1: Melia is one of the best. You know, some filmmakers
Speaker 1: were just gob smacked at the idea of being able
Speaker 1: to record a performance that could be played back endlessly,
Speaker 1: or at least, you know, until the film war out.
Speaker 1: Mellier was getting really experimental. Now, remember in ye olden
Speaker 1: days everything was shot on film, and some of you
Speaker 1: might not really know that much about film because things
Speaker 1: have changed so so quickly. You know, back when I
Speaker 1: was a young lad, hardly anyone had access to digital cameras.
Speaker 1: That we all had access to film cameras. Film is
Speaker 1: a strip of plastic that's coated with chemicals that are photoreactive.
Speaker 1: That means that when they are exposed to light, they
Speaker 1: undergo a chemical reaction. This opens up the opportunity to
Speaker 1: do stuff like double exposures. Now, a double exposure is
Speaker 1: when you expose the same piece of film to light
Speaker 1: more than once. Uh. Now, sometimes that would happen by accident.
Speaker 1: You know, if you had a film camera, you might
Speaker 1: accidentally take a photo on a frame that had already
Speaker 1: been exposed to light once before, and you would end
Speaker 1: up with sometimes an unsettling image. You know, it might
Speaker 1: look like there was kind of a ghostly figure appear
Speaker 1: ring in a separate scene, and it could be really
Speaker 1: off putting. Sometimes it was just a mass, depending on
Speaker 1: what you were taking images of. But if you did
Speaker 1: it purposefully and carefully, you could achieve really interesting effects.
Speaker 1: And Malier created a film called The One Man Band
Speaker 1: that used seven exposures and it's a silent film. It's
Speaker 1: a short silent film. You can actually find it on YouTube.
Speaker 1: You can do a search for the one man band
Speaker 1: and watch it. And it starts with a man walking
Speaker 1: in front of seven chairs that are set out on
Speaker 1: a stage there, side by side, and he points out
Speaker 1: those seven chairs very uh, in a very exaggerated fashion,
Speaker 1: and then he sits in the first chair on the
Speaker 1: left side of the view, and then stands up, but
Speaker 1: a figure of him remains seated in that chair number one.
Speaker 1: The standing version of him moves over one, sits in
Speaker 1: chair number two, then stands up. Copy of him remains
Speaker 1: in chair number two. So now you've got pies of
Speaker 1: them in chairs one and two, and a standing version
Speaker 1: that moves to three. And he does this all the
Speaker 1: way down, so that there are ultimately seven of him,
Speaker 1: so presumably one original and six copies. And not only that,
Speaker 1: but each time he stands up, the one that's in
Speaker 1: the chair as holding a different musical instrument, and uh,
Speaker 1: the one in the middle actually stands up and conducts
Speaker 1: the other six as they appear to play music against
Speaker 1: the silent films. You don't actually hear anything, but it
Speaker 1: is a clever show of how this multiple exposure approach
Speaker 1: would allow you to do in camera effects. Now that
Speaker 1: technique is interested, but it is limited. You know, you
Speaker 1: can technically clone a person on screen using multiple exposure,
Speaker 1: but you know, if they cross paths, you get this
Speaker 1: kind of ghostly effect, and you would have to do
Speaker 1: it with right lighting so that no one appeared kind
Speaker 1: of like transparent, unless I guess that's what you were
Speaker 1: going for, in which case, go for it. But you know,
Speaker 1: the next example is one that more closely ties into
Speaker 1: what the volume and the A R wall are doing.
Speaker 1: And it is called background projection. And yeah, it's it's
Speaker 1: exactly what it sounds like. It's projecting a background. So
Speaker 1: imagine here's a very typical use of it. Imagine that
Speaker 1: you've got a sound stage, right, you can control everything
Speaker 1: in the sound stage. It's an ideal environment for shooting
Speaker 1: a film. You don't have to worry about being on
Speaker 1: location and dealing with things like whether or noise or
Speaker 1: anything like that. So you're on your sound stage and
Speaker 1: you want to shoot a sequence where your characters are
Speaker 1: in a car and they're traveling somewhere. Well, clearly you
Speaker 1: can't actually have the car driving around on the sound
Speaker 1: stage at least not more than a few feet. So
Speaker 1: one way you would get around this is you would
Speaker 1: set up your car. You would have your actors sitting
Speaker 1: in the car, and behind the car, to the side
Speaker 1: of the car, whatever angle you were looking at, you
Speaker 1: would set up screens or plates, and upon that you
Speaker 1: would project video or really film of movement that would
Speaker 1: be in the right angle compared to the vehicle's orientation.
Speaker 1: So this is something that would require a lot of uh, finagling,
Speaker 1: just to make sure you had it right, otherwise it
Speaker 1: would be very off putting. Your camera would need to
Speaker 1: be at the right angle, pointed at the car, and
Speaker 1: the screens would me need to be at the right angle,
Speaker 1: and the footage would need to be shot at the
Speaker 1: right angle to give the illusion that the people saying
Speaker 1: in the car actually traveling down like a country lane
Speaker 1: or something like that. Meanwhile, you film the whole thing
Speaker 1: in your film camera, so you're actually shooting not just
Speaker 1: live actors, but you're shooting pre shot filmed footage that's
Speaker 1: playing in the background. It wasn't perfect, you know. You
Speaker 1: can definitely tell when this is used. If you watch
Speaker 1: any old film or old television series, you can pick
Speaker 1: out when this happens when characters are in a car
Speaker 1: and you just look and you say, yeah, that that
Speaker 1: that is clearly background projection. And obviously if you didn't
Speaker 1: have the alignment right, it would be kind of weird too. Right.
Speaker 1: Let's say that the angle you've chosen to shoot your
Speaker 1: scene is kind of a three quarter profile type thing,
Speaker 1: but the angle of the footage is straight on going
Speaker 1: down the street, Well, it would look like the car
Speaker 1: was somehow traveling out of its alignment down the road.
Speaker 1: So everything had to fit or else the illusion was
Speaker 1: completely broken. And even when the illusion wasn't completely broken,
Speaker 1: it's still pretty obvious. So if you watch films from
Speaker 1: the so called Golden Age of Hollywood, you're probably gonna
Speaker 1: see a lot of background projection. It was very, very popular,
Speaker 1: and you can usually tell right away the background tends
Speaker 1: to be extra grainy compared to stuff that's going on
Speaker 1: in the foreground. Film has a grainy quality to it,
Speaker 1: So shooting film on film gets you know, that grain
Speaker 1: equality gets amplified, right, because you're picking up the grain
Speaker 1: of the background film. Plus it's the grain of the
Speaker 1: actual film you're using, So it's stacks and there are
Speaker 1: tons of other camera tricks that I could talk about
Speaker 1: from the classic film age, but most of it doesn't
Speaker 1: actually actively apply to the volume and the A R wall. Instead,
Speaker 1: we should specifically talk about techniques where you're trying to
Speaker 1: create foreground action and a manufactured background, because early on
Speaker 1: filmmakers would create elaborate backdrops to serve as a background.
Speaker 1: And again that meant that you could shoot on a
Speaker 1: sound stage, so you had your controlled environment and it
Speaker 1: would cut back on a lot of things that could
Speaker 1: potentially waste your time. You have to also remember in
Speaker 1: the film days, you had a limited amount of film
Speaker 1: that you had at your disposal, right, and if you
Speaker 1: ran out a film, you would have to purchase more.
Speaker 1: So you couldn't just shoot and shoot and not care
Speaker 1: whether or not it was working. You had to care
Speaker 1: a lot because it directly related to how much money
Speaker 1: you had to spend in order to make the movie.
Speaker 1: So sound stages were preferred in many cases to shooting
Speaker 1: on location because of that. So if you could create
Speaker 1: a really detailed background that you know due to how
Speaker 1: the camera is going to focus on characters in the foreground,
Speaker 1: it can pass as realistic. Right. You can have a
Speaker 1: painting in the background and it's fine. Uh. Some of
Speaker 1: these backgrounds called mats were pretty convincing. Some we're not.
Speaker 1: But a great matt painting is something that I always
Speaker 1: enjoy seeing in a movie. And once in a while
Speaker 1: you'll get a filmmaker that uses matt paintings today, typically
Speaker 1: because they want to evocus specific sort of reaction from
Speaker 1: their audience. It's not not used as a common technique today.
Speaker 1: It's almost more as a stylistic choice and the word
Speaker 1: that would be used for a lot of different techniques
Speaker 1: and equipment to achieve the goal of come binding image
Speaker 1: elements from separate sources into a final image. We'll talk
Speaker 1: about more of that after we come back from this
Speaker 1: quick break, Okay. So ultimately, matts can be different things,
Speaker 1: Like it could be a little bit confusing when you're
Speaker 1: talking about mats in filmmaking because I don't know about you,
Speaker 1: but I'm used to using a word to mean something specific,
Speaker 1: and matt is usually used more as a way to
Speaker 1: achieve a specific result. So a matt could be a
Speaker 1: mask that sits on the camera that shields the film
Speaker 1: from being exposed to light beyond a certain point. So,
Speaker 1: in other words, it's it's like you're you're keeping a
Speaker 1: blackout area so that film doesn't get exposed to light.
Speaker 1: And then you're shooting another scene where it's the complementary scene,
Speaker 1: the couple of mimmentary footage I guess I should say,
Speaker 1: and has an opposite mask, right, It masks every anything
Speaker 1: else and leaves open the part that had been covered
Speaker 1: in the first mask. And then you combine these two
Speaker 1: sources of film together to create your final image. Uh,
Speaker 1: here's an example. Let's say that you're shooting a scene
Speaker 1: in which your hero is in an alley and their
Speaker 1: back is against the brick wall of a shop, and
Speaker 1: the hero is close to the corner of the front
Speaker 1: of the shop, and so the way you positioned the camera,
Speaker 1: you can see the hero in the shadows of this alley,
Speaker 1: and you can also see on the you know what's
Speaker 1: around the corner. And let's say you wanted a really
Speaker 1: interesting shot of a foggy street. Well, you're brick building
Speaker 1: is a facade that's inside the sound stage, and what
Speaker 1: you've done is you've created an amazing Matt painting of
Speaker 1: a foggy street. And so you've got a mask on
Speaker 1: your camera that blocks everything that's around the corner of
Speaker 1: the wall, so that you can fill it in with
Speaker 1: something its later. And then you shoot a a shot
Speaker 1: of the matt painting where you've got a mask that
Speaker 1: blocks out where the brick wall would be and your hero,
Speaker 1: and then you combine these two elements afterward to create
Speaker 1: the actual finished effect where you have your hero against
Speaker 1: the brick wall in the shade and the alley and
Speaker 1: around the corner you see this foggy street. That's a
Speaker 1: very super simple example. The mats in that case are static.
Speaker 1: The camera is locked down. You can't move the camera
Speaker 1: at all in that kind of shot. But let's say
Speaker 1: that you do want to create a shot in which
Speaker 1: people are moving, or maybe stuff in the background is
Speaker 1: moving with relation to the people. Uh Like, Let's say
Speaker 1: that you also want a shot where our hero is
Speaker 1: stepping beyond the corner of the building to look down
Speaker 1: this foggy street that doesn't actually exist in the sound stage,
Speaker 1: or maybe you want the camera to be able to
Speaker 1: move to pan and and dolly and that sort of off. Well,
Speaker 1: in that case, you need what is called a traveling matt.
Speaker 1: Also a shout out to those of you who are
Speaker 1: familiar with Fraggle Rock. That show used a few different
Speaker 1: technical film terms in it. Uncle traveling Matt was named
Speaker 1: after this particular thing in filmmaking. Gobo is another example,
Speaker 1: but I digress. There are a couple of different approaches
Speaker 1: to traveling mats. You know, it's not again just like
Speaker 1: the word matt. There are multiple examples of it, but
Speaker 1: the one that I referred to earlier was one where
Speaker 1: you use a green or a blue screen. And if
Speaker 1: you're talking about traditionally in the old days of filmmaking,
Speaker 1: it was typically a blue screen. This is called chroma keying,
Speaker 1: and the basic idea is simple to understand from a
Speaker 1: very high level, but it's actually really tricky to make
Speaker 1: it work properly in the film days in particular. So
Speaker 1: you take a solid color such as blue, and you
Speaker 1: use that to serve as the background as you're shooting
Speaker 1: your principal action. You make sure none of your performers
Speaker 1: have any closer props that are that color of blue
Speaker 1: really blue at all if you can avoid it, and
Speaker 1: you film your actors in front of this blue screen,
Speaker 1: and then you pair that with other footage that's meant
Speaker 1: to be the background. It effectively replaces all the blue
Speaker 1: parts of the background with this other footage that you've
Speaker 1: shot or created. Now, doing this was not easy back
Speaker 1: in the film days. To accomplish a traveling matt, you
Speaker 1: had to shoot on three separate strips of film simultaneously.
Speaker 1: One strip would be for red, one for blue, and
Speaker 1: one for green. You would take the negative image for
Speaker 1: the blue strip of film and the positive image for
Speaker 1: the green strip of film to combine those to create
Speaker 1: a solid matt. Then you could composite that with the
Speaker 1: background footage that have been shot against a separate blue screen,
Speaker 1: and these separate strips of film would be fed to
Speaker 1: an optical printer to create the composite final image and
Speaker 1: the whole system you would be calling this keying out.
Speaker 1: You'd be keying out in edges to replace with some
Speaker 1: other footage. And it worked, but you typically could see
Speaker 1: the effects on screen. Um, if people were really, really,
Speaker 1: really meticulous about it, it was pretty effective. But largely
Speaker 1: you would look at this and think like, oh, no,
Speaker 1: I know how they did that. I'm not necessarily completely
Speaker 1: pulled out of the action. But depending on how bad
Speaker 1: the effect was, it could do that. So you know what,
Speaker 1: when you see it, there's this kind of there can
Speaker 1: be a hard line between the borders of an actor
Speaker 1: and the background, and you can tell that the people
Speaker 1: who are in the scene aren't really in that environment,
Speaker 1: whatever that environment might be. Now. Back in the day,
Speaker 1: movies that did use chroma key typically use blue screens
Speaker 1: because film itself has a blue emulsion layer in it.
Speaker 1: And when the transition to digital cameras happened, there was
Speaker 1: a switch to green screens, which were more effective with
Speaker 1: digital cameras, and you could set up enormous green screens
Speaker 1: outside even and not have to worry about the camera
Speaker 1: catch blue sky and creating an opportunity for the chroma
Speaker 1: key process to bleed over um. In that case, obviously,
Speaker 1: you don't want people wearing green stuff because it would
Speaker 1: be the same issue where the whatever the background footage
Speaker 1: would be replacing the green on the character or proper whatever.
Speaker 1: Green screens gave filmmakers the chance to transport characters to
Speaker 1: places that don't exist without having to actually build all
Speaker 1: the darn fantastical landscapes themselves. But as I mentioned earlier,
Speaker 1: this also puts a burden on the actors as they
Speaker 1: can't actually see whatever it is that's going to show
Speaker 1: up in the final film. They have to imagine it.
Speaker 1: That's not always easy when you're also trying to hit
Speaker 1: your mark and say your lines in a way that's,
Speaker 1: you know, good. And obviously the more unusual the location
Speaker 1: or the characters that are surrounding you are, the more
Speaker 1: it's asking the actor to use their imagination. And it
Speaker 1: could very well be that the actor is totally off
Speaker 1: base when they're imagining what the filmmaker is imagining. It
Speaker 1: could that those two visions don't line up at all.
Speaker 1: It creates problems and chrome Mickee also has some limitations
Speaker 1: that you have to light the shot really well or
Speaker 1: else the effects won't look as good. And by well,
Speaker 1: I mean that that can limit how much you can
Speaker 1: play with light and shadow. So you can create incredible backdrops,
Speaker 1: and you might not be able to to tell the
Speaker 1: difference that you know that backdrop wasn't actually there, but
Speaker 1: you might not be able to light the scene in
Speaker 1: a way that's really compelling. It is possible to do,
Speaker 1: but it's pretty complicated because you have to very carefully
Speaker 1: light your foreground and your background separately, and then you
Speaker 1: have to keep in mind stuff like reflections and shadows.
Speaker 1: It's just it's really hard to pull off, and it's
Speaker 1: very easy to make mistakes, and for the very keen
Speaker 1: eyed movie goers out there, those mistakes end up being obvious,
Speaker 1: not not to me so much. I typically don't notice
Speaker 1: unless it's unforgivably obvious. Now, what if you could combine
Speaker 1: elements like background for action, traveling mats and a controlled
Speaker 1: studio environment. What if your actors could actually see the amazing,
Speaker 1: perhaps even entirely virtual locations that their characters in habit,
Speaker 1: which eliminates the need for them to rely solely on
Speaker 1: their imaginations. What if you could leverage the benefits of
Speaker 1: computer generated environments and put real world actors, props, and
Speaker 1: set pieces inside it. What if these environments were generating
Speaker 1: light that could interact with the stuff that's really there
Speaker 1: in front of the camera, So you get reflections and
Speaker 1: shades of light on the performers because it's not that
Speaker 1: they've been virtually put into this environment, it's that the
Speaker 1: environment is virtually around the actors. Here's where we get
Speaker 1: to the volume and the A R wall. So these
Speaker 1: are systems that use L E D screens, huge l
Speaker 1: E D screens, much much bigger than your television and
Speaker 1: they are positioned around and above a studio area, and
Speaker 1: it creates a panoramic view that partially surrounds the actors
Speaker 1: in a scene. So with the A R wall, it's
Speaker 1: like a very large arc that also has a ceiling
Speaker 1: with screens in it. Uh. If it's the volume, it
Speaker 1: ends up being almost a complete circle, also with a
Speaker 1: ceiling that has led screens in it. So if you're
Speaker 1: familiar with the Star Trek Next Generation television program, then
Speaker 1: you are also familiar with the holiod Deck. The Holid
Speaker 1: Deck is a room aboard the starship Enterprise and presumably
Speaker 1: other ships too, And in this Holid Deck you could
Speaker 1: go in and the ship's computer would generate an entire
Speaker 1: virtual world within that room, complete with illusions that you
Speaker 1: could sense beyond just seeing them. You could smell things,
Speaker 1: you could even pick stuff up, and if you were
Speaker 1: a Commander Riker, you could hit on ai generated women
Speaker 1: in a way that was totally creepy and weird and
Speaker 1: definitely has not gotten better with age. Uh. Also, the
Speaker 1: Holidack would malfunction in pretty much every other episode had
Speaker 1: appeared in which left you wondering why anyone would ever
Speaker 1: go inside the darn thing. In the first place. But
Speaker 1: the point being the holidack would create this virtual environment
Speaker 1: around you so that you had this immersive experience. Well,
Speaker 1: the volume and the A R wall are kind of
Speaker 1: like the holidack, but obviously with huge limitations, Like the
Speaker 1: holidack could move stuff around. Uh, it could adjust things
Speaker 1: so that people were not gonna walk into a wall.
Speaker 1: Somehow they always happen to know where the wall was.
Speaker 1: I don't know how that happened, because it looks like
Speaker 1: you're in this you know, immersive environment. But anyway, you
Speaker 1: wouldn't walk into a wall. You could actually pick things
Speaker 1: up because this virtual world could you know, replicate stuff
Speaker 1: so that there were physical things for you to contact. Obviously,
Speaker 1: we can't do that in our real world, and we're
Speaker 1: not at a point where we can, you know, manipulate
Speaker 1: the environment in such a way that you never get
Speaker 1: to the edge. You will eventually get to the edge,
Speaker 1: because that's how reality works. So on a base level,
Speaker 1: what is going on is that these systems are projecting complex, dynamic,
Speaker 1: and immersive backgrounds. Now, all three of those things are important, right,
Speaker 1: They are complicated backgrounds. It's not just a simple green screen.
Speaker 1: Although they can do that too. They are dynamic in
Speaker 1: the sense that it doesn't have to be just a
Speaker 1: static background. It's not like a matte painting. It can
Speaker 1: have It can incorporate moving things that you could have
Speaker 1: shot other footage and use that projected within these environments.
Speaker 1: So if you wanted to, you could film extras milling around,
Speaker 1: you know, a medieval battleground after a war has been fought,
Speaker 1: and then just use that as background against the action
Speaker 1: of your characters in the foreground if you wanted to.
Speaker 1: This has some obvious benefits. One is that you don't
Speaker 1: need to go shoot on location and a world WARLD
Speaker 1: travel restrictions can pop up quickly due to stuff like
Speaker 1: COVID and now monkey pocks. That's a huge asset to
Speaker 1: a production company because just think for a moment about
Speaker 1: the logistics that are required for you to shoot on location,
Speaker 1: particularly if you want to shoot someplace that's remote and
Speaker 1: maybe in another country, because you want a place that
Speaker 1: doesn't have as much, uh, you know, identifiable landmarks that
Speaker 1: are on on site. You because you want to create
Speaker 1: this illusion of this this world, whether it's fantasy world,
Speaker 1: science fiction, whatever it might be. Well, just getting access
Speaker 1: to that location, getting the gear and the crew and
Speaker 1: the cast all to the right place. All of that
Speaker 1: requires huge amounts of work, their entire production departments dedicated
Speaker 1: to making that happen. Then once you get there, you
Speaker 1: might have to worry about stuff like weather and lighting
Speaker 1: and all that. So in a controlled studio environment, you
Speaker 1: don't have those issues. Right as long as people are
Speaker 1: testing clear so they're not, you know, coming down with
Speaker 1: COVID or anything like that, then you're good to go
Speaker 1: and you don't have to worry about relocating everybody halfway
Speaker 1: around the world. And of course a huge issue is
Speaker 1: that it starts to bring elements of pre production, production
Speaker 1: and post production together in one environment. I cannot stress
Speaker 1: what a huge change this is for filmmaking because it
Speaker 1: means that ultimately you don't need as much time to
Speaker 1: complete a project. And y'all time is money. So if
Speaker 1: you have a system where you can design and then
Speaker 1: project a a background onto these screens that your actors
Speaker 1: can actually see, that the camera can see that, you
Speaker 1: don't even have to change for it to appear. As
Speaker 1: as camera ready, it's it's to go, and then say
Speaker 1: the director says I want to change things. It gives
Speaker 1: you the opportunity to do that, so you can actually
Speaker 1: do visualization really close to production. And because those effects
Speaker 1: are already there, and the lighting effects and everything are
Speaker 1: already there, it reduces the amount of work you need
Speaker 1: to do in post production. That's phenomenal. Now, obviously this
Speaker 1: still requires a ton of work. It's it's not like
Speaker 1: it just all magically happens. You have to create the
Speaker 1: background images. That's I'm going to require a lot of work,
Speaker 1: both from an artistic point of view and a technical one. Uh.
Speaker 1: And it's not just that these environments can show digital
Speaker 1: images and act kind of like an LED version of
Speaker 1: a Matt painting. Uh. The A R wall in particular,
Speaker 1: has a really cool feature in it in which only
Speaker 1: sections of the wall that are in view of the
Speaker 1: camera are creating high resolution output. And and that's a
Speaker 1: very clever solution. So just imagine that this is how
Speaker 1: the old works. For a second, imagine that whenever you
Speaker 1: look around, only the stuff that's actually within your frame
Speaker 1: of you is well defined. In other words, the stuff
Speaker 1: that you see in focus is the only stuff that's
Speaker 1: in focus in the world. It's whatever you're looking at.
Speaker 1: Anything you're not looking at, like anything that's outside your
Speaker 1: frame of you is more hazy until you turn to
Speaker 1: look at it, at which point it instantly appears in
Speaker 1: clear focus. Now you can argue that the real world
Speaker 1: doesn't behave this way, But a philosopher would ask, are
Speaker 1: you sure about that? Are you are you absolutely sure
Speaker 1: about that? Or is it just that every time you
Speaker 1: look around the world snaps into focus and everything else
Speaker 1: kind of fades into this fuzziness. Well, that's the way
Speaker 1: the the A R wall works with a relation to cameras.
Speaker 1: I'll explain why that's really cool and important after we
Speaker 1: come back from this quick break. Okay, so we have
Speaker 1: seen this approach of making sure that only the thing
Speaker 1: that's within focus, that that that's being looked at is
Speaker 1: within focus, I guess I should say is important. We've
Speaker 1: seen that in the past. So, for example, VR engineers
Speaker 1: will often create headsets and software in which the stuff
Speaker 1: that is appearing directly in front of a person's eyes
Speaker 1: is in high resolution, but the stuff that's at the edges,
Speaker 1: you know, in the peripheral view of the user, can
Speaker 1: be in lower resolution. Because that's how our eyes work,
Speaker 1: that's how our vision works. That saves on compute power
Speaker 1: and it means that more resources can go to producing
Speaker 1: other stuff like a high frame rate. For example, if
Speaker 1: you're not looking at it, then it doesn't need to
Speaker 1: be in high resolution, it doesn't even have to be
Speaker 1: rendered at all. So the same can be true for
Speaker 1: the A R wall. The way they do this is
Speaker 1: that they pair tons of sensors together so that they
Speaker 1: can know where the camera is pointing at any moment.
Speaker 1: And by no, I mean like the system is aware
Speaker 1: of where the camera is looking at. It's not a
Speaker 1: human who's doing this. So each camera, like production camera,
Speaker 1: the camera that people are using to shoot an episode
Speaker 1: of Star Trek for example, has mounted on top of
Speaker 1: it a frame, and that frame holds some little spheres
Speaker 1: on it, which act is kind of like an orientation
Speaker 1: key for the camera sensors that are incorporated all around
Speaker 1: the A R wall. So these are little things that
Speaker 1: get can be digitally erased in production, but they are
Speaker 1: at all these different points within the A R wall frame.
Speaker 1: Those sensors are all picking up that that little series
Speaker 1: of orientation spheres that are mounted onto the cameras, and
Speaker 1: by interpreting the angles, a computer system knows where that
Speaker 1: camera is pointed at. Any given time and thus can
Speaker 1: dedicate the resources needed to generate high resolution for just
Speaker 1: the frame of vi for that camera. It's incredible compute
Speaker 1: power to do this right, to be able to detect
Speaker 1: where the camera's pointed, you know, how far in is
Speaker 1: it zoomed? How much of the screen do you need
Speaker 1: to bring into sharp focus? But that's how it works.
Speaker 1: And as you move the camera around you see this.
Speaker 1: You could see the images on the the led walls
Speaker 1: come into you know, tighter focus or fade if the
Speaker 1: camera has just moved away from them. It's pretty trippy
Speaker 1: to watch. And the backgrounds can actually move with the
Speaker 1: camera to which helps create the illusion that there's something
Speaker 1: definitely back there as opposed to a flat image. And
Speaker 1: this kind of relates to something that's called motion parallax. Now,
Speaker 1: first let's just talk about parallax in general. So, assuming
Speaker 1: that you have vision in both eyes, you experience parallax.
Speaker 1: And what parallaxes is the apparent difference in position of
Speaker 1: an object along different lines of sight. Now that's a
Speaker 1: fancy way of saying that where something appears to be
Speaker 1: changes depending upon your point of view. If you were
Speaker 1: to hold your finger up in front of your face
Speaker 1: and close one eye and look at your finger, and
Speaker 1: then you close the open eye and open the closed eye.
Speaker 1: It would look like your finger had changed positions, right,
Speaker 1: because you're looking at it from your other eye, which
Speaker 1: is obviously not in the same physical position as I
Speaker 1: number one. You know, our eyes are are located side
Speaker 1: by side, So if we do that, if we switch
Speaker 1: back and forth, closing one eye and opening the other,
Speaker 1: it'll seem as if our finger is jumping back and
Speaker 1: forth between two different positions. It's not really doing that, obviously,
Speaker 1: that's just what you're seeing based upon that particular line
Speaker 1: of sight, and our brains combine these two lines of
Speaker 1: sight to create the image that we use to kind
Speaker 1: of interpret the world around us and to do things
Speaker 1: like get an idea of how far away an object is.
Speaker 1: For example. Now, motion parallax is slightly different. Imagine that
Speaker 1: you've got two characters, one who is closer to the camera,
Speaker 1: one who is further back. And let's say that they're
Speaker 1: both walking from the left side of camera view to
Speaker 1: the right side, and they're both walking at the same speed. Well,
Speaker 1: the person in front will cross the frame faster because
Speaker 1: they're closer to us. Than the person who's further behind.
Speaker 1: So this also applies if you're moving the camera and
Speaker 1: the various components are standing still. If the person in
Speaker 1: the foreground and the person in the background are standing
Speaker 1: still and you're handing the camera across, then the person
Speaker 1: in front is going to cross the frame much faster. Right.
Speaker 1: This is stuff that we intuitively know because it's how
Speaker 1: we experience the world around us. Well, when you're creating
Speaker 1: a digital system that is standing in for a real background,
Speaker 1: sometimes you've gotta move stuff around and cheat in order
Speaker 1: for this impression to be realistic and immersive. So you
Speaker 1: can actually have the camera's movements and the backgrounds movements
Speaker 1: choreographed together, so that as you're moving the camera around,
Speaker 1: the background is actually moving slightly to in order to
Speaker 1: get the right effect. That's something you cannot do with
Speaker 1: a Matt painting, obviously, because a Matt painting is just
Speaker 1: going to be that static image forever and ever. Now,
Speaker 1: another benefit of this technology is, like I said, it
Speaker 1: provides lighting for a scene. So if a character like
Speaker 1: say the Mandalorian, is inside an environment that has lots
Speaker 1: of lights all around, the lights from the screen will
Speaker 1: reflect on the Mandalorian's shiny best car armor, so it
Speaker 1: creates a more immersive, realistic effect. If you have a
Speaker 1: close up on the Mandalorian's helmet as he's sitting in
Speaker 1: his spaceship and he's going into uh To, you know,
Speaker 1: hyper speed, well you'll see the reflections off his hell
Speaker 1: because they're actually projecting that hyperspeed imagery all around the
Speaker 1: set that has been built inside the volume. And again
Speaker 1: you don't have to do that in post. It's created
Speaker 1: right there in the camera shot. This is stuff that
Speaker 1: typically you would have to add afterward, but you have
Speaker 1: removed that step by including it in the moment when
Speaker 1: you're actually shooting the scene. Now, to make all of
Speaker 1: this happen, there have to be these big old computer
Speaker 1: systems in the background calculating things like the angle of
Speaker 1: view of the camera, the motion of the camera, how
Speaker 1: the motion of the background should match that. And obviously
Speaker 1: you have to actually create the assets that are going
Speaker 1: to be on display on those screens. That might be
Speaker 1: computer generated assets, in which case you have artists creating
Speaker 1: these things, building them in virtual engines which then get
Speaker 1: rendered onto the screens. Alternatively, you might incorporate some old
Speaker 1: school stuff like miniature work that's been mapped to a
Speaker 1: digital background that then it's projected onto these l e
Speaker 1: D screens or by these LED screens. I say projected on,
Speaker 1: it's really by because the the images coming from the
Speaker 1: l e ED is not being projected onto them. Or
Speaker 1: you might include camera footage from real locations and again
Speaker 1: that could be mapped onto the screens. So the first
Speaker 1: generation volume, the one that was used in Star Wars
Speaker 1: projects like the first season of The Mandalorian, was a
Speaker 1: seventy five ft diameter twenty one ft tall environment. They
Speaker 1: used the Unreal game Engine for initial pre viz. That's
Speaker 1: pre visualization and design, and I think that's pretty darn cool.
Speaker 1: It's really neat to see filmmakers leverage a tool that
Speaker 1: was meant for game development and use it to create
Speaker 1: film sets, digital virtual film sets that then could be
Speaker 1: displayed on these these screens. Obviously, there are a lot
Speaker 1: of other software packages, many of which were specifically created
Speaker 1: for the filmmaking industry, many of them created by Industrial
Speaker 1: Light and Magic itself, which is the entity that created
Speaker 1: the Volume UM in order to create the graphics that
Speaker 1: are seen within these environments and ultimately on the final product.
Speaker 1: Whether that's a film or series. The volume used seven
Speaker 1: machines to create the compute power to provide the graphics
Speaker 1: and dynamics necessary to make it all work for that
Speaker 1: season one. For the second season, the Mandalorian Industrial Light
Speaker 1: and Magic created Stagecraft two point oh and Stagecraft two
Speaker 1: point oh handles rendering and projection along with the software
Speaker 1: and hardware that makes it all work together, and it
Speaker 1: also works with other products like Helios, which is a
Speaker 1: video rendering and lighting package specifically made for the film industry.
Speaker 1: Beyond that, the nature of the technology means that changes
Speaker 1: can be made relatively quickly. I mean, computer generated assets
Speaker 1: can be manipulated. So if you are looking at a
Speaker 1: background and you think, you know, I wish that mountain
Speaker 1: were larger, you could actually make the mountain larger. It's
Speaker 1: a digital asset. You can change the shape and size
Speaker 1: of it. So they can be enlarged, they can be shrunk,
Speaker 1: they can be repositioned, changed in various ways, all near
Speaker 1: real time. Like it's not it's not instantaneous, but it's
Speaker 1: super fast. So if a cinematographer gets in the space
Speaker 1: and they see the virtual background, they say, you know what,
Speaker 1: that craggy tree would actually look better if we put
Speaker 1: it closer to this little hill over here, it will
Speaker 1: make my shot really work. Then people working on the
Speaker 1: film can actually jump in onto the virtual environment, make
Speaker 1: those changes, and then project the new version of it
Speaker 1: within minutes, as opposed to this being something that takes
Speaker 1: days or weeks to do and post. Now, these capabilities
Speaker 1: still do have their limitations. While the areas inside the
Speaker 1: volume and the A R wall are are pretty large,
Speaker 1: you still do have walls, just like the enterprise is
Speaker 1: holiday and there's no way for a computer to shift
Speaker 1: people inside those environments so that they don't have to
Speaker 1: worry about that. And for that reason, it could be
Speaker 1: a little challenging to choreograph like epic action sequences within
Speaker 1: these still limited spaces, and you might have seen some
Speaker 1: recent examples of action scenes I'm thinking specifically of the
Speaker 1: Book of Boba Fette that didn't quite live up to
Speaker 1: the term epic. These led background environments represent powerful new
Speaker 1: tools for filmmakers, but just like other tools like three
Speaker 1: D and c g I, they have to be used
Speaker 1: properly to contribute to the experience of watching a series
Speaker 1: or watching a film, rather than detract from them. I
Speaker 1: think that the volume and the A R wall both
Speaker 1: give filmmakers some of the same tool sets that people
Speaker 1: who have been working in in c G animation have
Speaker 1: had at their disposal forever. So if you look at
Speaker 1: Pixar and the way they create films, the way they
Speaker 1: create virtual environments, a lot of the same tools that
Speaker 1: they are using are the ones that we're now seeing
Speaker 1: used in live action filmmaking. At the point where you're
Speaker 1: actually shooting and where those elements are there, now, you
Speaker 1: don't have all the advantages that you would with c G.
Speaker 1: So Pixar, for example, they could technically create a camera
Speaker 1: that could do all sorts of things that a physical
Speaker 1: camera never could, right, they are not limited by the
Speaker 1: rules of real world physics when it comes to that.
Speaker 1: Pixar doesn't typically do that because they learned very quickly
Speaker 1: that if you start violating basic rules, uh, audiences get
Speaker 1: pulled out of the action because they say, like, well,
Speaker 1: that's impossible. It's it's you have to work somewhat within
Speaker 1: the rules in order for it to to work for audiences.
Speaker 1: But it doesn't matter for the volume more for the
Speaker 1: A R wall, because in both cases you're still using
Speaker 1: real physical cameras to shoot real physical people the background images.
Speaker 1: You could mess with that, but then that doesn't make
Speaker 1: a whole less sense either unless you're doing something really trippy,
Speaker 1: like I could see maybe a future Doctor Strange movie
Speaker 1: using that sort of thing to kind of be mind
Speaker 1: bindy and create something that is absolutely impossible, and yet
Speaker 1: the actors who are there would be able to see
Speaker 1: it and react to it. Um, that is something where
Speaker 1: I could see that work. But otherwise, what I think
Speaker 1: is really cool is that it does give those tools
Speaker 1: that that c G animators have had at their disposal
Speaker 1: forever and put it into real live action filmmaking, as
Speaker 1: well as that whole concept of bringing pre and post
Speaker 1: production closer to production itself so that you are able
Speaker 1: to make these decisions. Directors can focus more on getting
Speaker 1: the performance they want and not worry as much about
Speaker 1: whether or not it's going to the way they imagined
Speaker 1: once you finally get all the effects added in, because
Speaker 1: that's that's the big issue with like shooting against screen screen.
Speaker 1: You might have an idea in your head what's supposed
Speaker 1: to look like, but until all the effects departments have
Speaker 1: put in their work and then keyed out the the
Speaker 1: shots that you got on set with all the green screens,
Speaker 1: you don't really know. I would argue that that perhaps
Speaker 1: is one reason why I find the third act of
Speaker 1: most Marvel films to be really messy, Like they just
Speaker 1: look too chaotic and video game ish for me almost
Speaker 1: every case. So I feel like these sorts of tools
Speaker 1: could potentially help with that. But then you also have
Speaker 1: to remember you have to work within the constraints of
Speaker 1: the space. You know, you can't go beyond the borders
Speaker 1: of the volume or the A R wall or else
Speaker 1: you lose the benefit of those and you break immersion. Anyway,
Speaker 1: Still really cool, really interesting to see how this convergence
Speaker 1: of technologies, from the super high resolution LED screens that
Speaker 1: are capable of making backgrounds that don't look like they're screens,
Speaker 1: like it looks like these are real locations, all the
Speaker 1: way to the real time computation of measuring things like
Speaker 1: camera angle and displaying the differences in the background in
Speaker 1: real time so that you get the the realistic effects
Speaker 1: you want. All of that is phenomenal to me. There are,
Speaker 1: by the way, production features on both the A R
Speaker 1: wall and on the volume that you can find online.
Speaker 1: I recommend if you're interested in these, you watch them
Speaker 1: because that will give you a little more information. They
Speaker 1: come across a bit marketing ish to me, especially the volume,
Speaker 1: because it's really industrial light and magic saying hey, we
Speaker 1: got this cool tool. If you're making a movie, call us,
Speaker 1: But it does give you more insight into what they
Speaker 1: are doing and why it's important. Anyway, thank you Bronson
Speaker 1: for your request. If any of you out there have
Speaker 1: a subject that you would like me to tackle, you
Speaker 1: can get in touch with me and let me know.
Speaker 1: There are a couple of ways doing that. One is
Speaker 1: you can hop on over to the I Heart Radio app.
Speaker 1: It's free to download. Navigate over to the tech Stuff podcast.
Speaker 1: There's a little microphone icon there. You can leave a
Speaker 1: thirty second message for me and let me know that way,
Speaker 1: or you can always pop over onto Twitter use the
Speaker 1: handle tech stuff hs W to leave your request like
Speaker 1: Bronson did. And I look forward to seeing those and
Speaker 1: I'll talk to you again really soon. Text Stuff is
Speaker 1: an I Heart Radio production. For more podcasts from my
Speaker 1: Heart Radio, visit the i Heart Radio app, Apple Podcasts,
Speaker 1: or wherever you listen to your favorite shows.