WEBVTT - TechStuff Classic: How Motion Capture Works 0:00:04.240 --> 0:00:07.240 Welcome to tech Stuff, a production of I Heart Radios 0:00:07.320 --> 0:00:14.080 How Stuff Works. Hey there, and welcome to text Stuff. 0:00:14.200 --> 0:00:18.119 I'm your host, Jonathan Strickland. I'm an executive producer with 0:00:18.239 --> 0:00:21.760 I Heart Radio and I love all things tech and 0:00:21.800 --> 0:00:25.000 it's time for another classic episode of tech Stuff. This 0:00:25.079 --> 0:00:29.680 episode originally published on December nineteen, two thousand and twelve. 0:00:30.080 --> 0:00:34.360 It is titled how Motion Capture works and I love 0:00:34.479 --> 0:00:36.960 mo cap when it's done well. When mo cap has 0:00:36.960 --> 0:00:41.440 done well, you can get some phenomenal performances translated into 0:00:41.520 --> 0:00:44.720 different types of media, including video games and c g 0:00:44.920 --> 0:00:49.080 I films. It is an amazing tool and Chris Pullett 0:00:49.159 --> 0:00:52.360 and I break down how it works. So let's listen 0:00:52.360 --> 0:00:56.200 in on this classic episode. So today we thought we'd 0:00:56.240 --> 0:01:00.600 talk a bit about a type of performance that is 0:01:00.800 --> 0:01:06.319 relatively new as far as performance goes. Uh, something that uh, 0:01:06.360 --> 0:01:09.640 I guess this falls into our movie making category, but 0:01:09.760 --> 0:01:12.200 it's also something that's been used in things like video 0:01:12.319 --> 0:01:17.759 games and and other forms of media as well. Motion capture. Yeah, 0:01:17.800 --> 0:01:21.440 you'll even see it in UH in sports. They've been 0:01:21.680 --> 0:01:24.240 talking about this for a while now. And if you've 0:01:24.280 --> 0:01:30.680 ever seen the making of a a video or a game. Um, 0:01:30.800 --> 0:01:34.640 or you know even in sports rehabilitation, in medicine, Um, 0:01:35.080 --> 0:01:38.120 they where the people are wearing dots, little white dots 0:01:38.120 --> 0:01:42.520 all over their clothing and sometimes their faces and hands. Um, 0:01:42.560 --> 0:01:45.319 that's probably what they were doing. Either that or they 0:01:45.400 --> 0:01:48.880 just really like stickers. Yeah, I mean who doesn't. I 0:01:48.920 --> 0:01:52.600 remember being very competitive in elementary school in order to 0:01:52.600 --> 0:01:55.600 get a sticker. And also this is a tangent but 0:01:55.680 --> 0:01:59.400 a true story. I got a gold star sticker, uh 0:01:59.480 --> 0:02:02.760 just last month, as from Tracy, the head of our 0:02:02.880 --> 0:02:07.720 our site. So anyway, um, yeah, motion capture. Actually, there 0:02:07.760 --> 0:02:10.800 are a lot of different terms that you can use 0:02:11.320 --> 0:02:14.960 in this in this realm uh, motion capture am CAP 0:02:15.080 --> 0:02:17.360 is probably the one I hear the most frequently, but 0:02:17.400 --> 0:02:23.160 also things like performance animation, performance capture, digital puppetry, real 0:02:23.240 --> 0:02:26.800 time animation, motion scanning, which is really more of a 0:02:26.840 --> 0:02:30.800 proprietary thing, but these are The concept is pretty much 0:02:30.840 --> 0:02:33.320 the same across the board. The idea is to capture 0:02:33.919 --> 0:02:38.720 the physical representation of something and then converted into a 0:02:38.840 --> 0:02:42.560 virtual format. So usually it's something that's in motion, but 0:02:42.600 --> 0:02:45.400 it's not always that way. Uh. Since you know we're 0:02:45.400 --> 0:02:49.200 talking about motion capture, that makes sense, but you're trying 0:02:49.240 --> 0:02:53.360 to get uh, translates something that is moving through real 0:02:53.480 --> 0:02:58.440 space into a digital format. And Uh, there's different ways 0:02:58.440 --> 0:02:59.800 to do this. I mean you could do it the 0:03:00.040 --> 0:03:02.400 really hard way, which is where you study something and 0:03:02.400 --> 0:03:05.640 then you try to recreate it, uh, either by hand 0:03:05.760 --> 0:03:09.200 or by or digitally, you know, by by programming movements 0:03:09.280 --> 0:03:13.079 into an animated figure. But this is an idea that 0:03:13.160 --> 0:03:16.839 kind of takes that step out where you are directly 0:03:16.960 --> 0:03:20.800 porting the movements uh, something is making within physical space 0:03:20.840 --> 0:03:24.840 into virtual space. Yeah, there was an early technique, UM, 0:03:24.880 --> 0:03:26.600 And of course this is this is all an attempt 0:03:26.680 --> 0:03:31.359 to to get as real as you can with with animation. UM. 0:03:32.040 --> 0:03:35.880 And one of the earlier techniques that that was sort 0:03:35.880 --> 0:03:40.000 of a predecessor to this is called rotoscoping. Uh. Ralph 0:03:40.000 --> 0:03:43.040 Box She's Lord of the Rings had a lot of 0:03:43.160 --> 0:03:47.160 rotoscoping in it. Well, what happens is UM, in that 0:03:47.200 --> 0:03:50.480 case is that a real uh, a real human being 0:03:50.800 --> 0:03:53.760 goes through the motions and they act through the parts 0:03:54.040 --> 0:03:55.880 that are that you're going to see in the animation, 0:03:56.000 --> 0:03:59.400 and they shoot that on film. Yes, yes, and then 0:03:59.640 --> 0:04:03.120 the the animators basically are looking at that and are 0:04:03.200 --> 0:04:06.240 drawing more or less on top of that, they see 0:04:06.280 --> 0:04:08.480 a projection of that, and they are drawing, uh, the 0:04:08.520 --> 0:04:13.040 animation over that to capture the way that person's body looks. 0:04:13.600 --> 0:04:17.240 And this this was famous. You know, the Disney studios 0:04:17.240 --> 0:04:19.440 were famous for this. We're studying models and then they 0:04:19.480 --> 0:04:23.279 would do the rotoscoping technique to to try to make 0:04:23.480 --> 0:04:27.200 their uh, their characters look more realistic. Yeah, and there 0:04:27.200 --> 0:04:30.600 are some artists, like I said, like Bakshi who famously 0:04:30.920 --> 0:04:35.400 would leave the film image as part of the animation, 0:04:35.520 --> 0:04:38.400 so that you had this this weird effect where the 0:04:38.480 --> 0:04:42.120 thing you were looking at was part uh well quote 0:04:42.160 --> 0:04:46.680 unquote real image and part animated image, which was it 0:04:46.720 --> 0:04:49.720 was an artistic choice, uh, definitely something that was not 0:04:50.120 --> 0:04:53.440 meant to to necessarily fool you into thinking, oh, well, 0:04:53.440 --> 0:04:55.960 that animated character is moving very realistically. It was done 0:04:56.000 --> 0:04:58.920 on purpose, but it was. That's what I always think 0:04:58.960 --> 0:05:01.280 of when I think rotoscope as I just think of 0:05:01.320 --> 0:05:03.560 the different box sheet films, but in particular I think 0:05:03.560 --> 0:05:06.760 of his Lord of the Rings adaptation, um, which, as 0:05:06.800 --> 0:05:12.080 I recall, ended halfway through the Two Towers. So anyway, 0:05:12.120 --> 0:05:14.520 that's just bringing back memories. But yeah, that was that 0:05:14.600 --> 0:05:20.080 was sort of a precursor to motion capture motion capture itself. 0:05:20.080 --> 0:05:24.799 There are many different ways of achieving this. For example, 0:05:25.040 --> 0:05:29.400 there were it's not used very frequently now, but there 0:05:29.400 --> 0:05:34.120 were mechanical systems where you had sensors that would be 0:05:34.160 --> 0:05:39.599 attached to specific joints UH that would relay movement. And 0:05:40.000 --> 0:05:42.200 usually it was kind of like a like an actor 0:05:42.200 --> 0:05:48.560 would wear a physical metallic skeleton type device that would 0:05:48.560 --> 0:05:52.279 have the sensors attached to the various joints, and as 0:05:52.320 --> 0:05:56.839 the actor moved, the sensors would register the changes in 0:05:56.960 --> 0:06:02.640 motion in this metallic skeleton and UH, and that would 0:06:02.680 --> 0:06:07.800 be relayed through usually cables to a computer system that 0:06:07.839 --> 0:06:11.640 would measure these or take the measurements from the sensors 0:06:11.640 --> 0:06:15.320 and translated into movements for the virtual character. UH. It's 0:06:15.400 --> 0:06:19.120 very limiting on this particular system. There was another one 0:06:19.160 --> 0:06:24.160 that was a little more versatile, which was used electro magnets. 0:06:24.800 --> 0:06:28.160 And in this case you talked about sensors that would 0:06:28.200 --> 0:06:31.720 be attached by really thick cables that again would go 0:06:31.839 --> 0:06:35.360 to a computer and there'd be a magnetic field and 0:06:35.560 --> 0:06:39.040 by moving through this magnetic field, the sensors would pick 0:06:39.120 --> 0:06:41.880 up alterations. You know, it would you know, moving through 0:06:41.960 --> 0:06:46.200 magnetic field, you would get little electrical changes. We we've 0:06:46.240 --> 0:06:51.000 talked a lot about electricity magnetism in general, moving through 0:06:51.400 --> 0:06:56.120 uh fluctuating a magnetic field can induce electricity through a conductor, 0:06:56.279 --> 0:07:00.440 or putting electricity through a conductor can induce a magnetic field. 0:07:00.640 --> 0:07:03.880 So anyway, by moving these sensors through the magnetic field, 0:07:04.080 --> 0:07:08.000 it would create these electronic fluctuations that would then be 0:07:08.080 --> 0:07:13.000 measured and translated into movement. And again this was a 0:07:13.200 --> 0:07:17.520 fairly effective way of picking up movements. It actually didn't 0:07:17.640 --> 0:07:20.600 use as many points of contact as the optical systems 0:07:20.600 --> 0:07:22.520 that we mostly think about, that was the kind that 0:07:22.600 --> 0:07:25.240 Chris was referring to early with all the dots on 0:07:25.280 --> 0:07:28.520 the person. Those systems tend to have lots and lots 0:07:28.600 --> 0:07:32.400 and lots of points of reference. The electro magnet ones 0:07:32.920 --> 0:07:36.400 didn't tend to have as many points of reference because 0:07:36.960 --> 0:07:40.360 the software side of it, because you know, we do 0:07:40.400 --> 0:07:42.560 have a hardware and a software side to this. The 0:07:42.640 --> 0:07:46.440 software side would assume that the joints that these sensors 0:07:46.440 --> 0:07:50.720 were attached to behaved the way they normally would in humans, 0:07:51.080 --> 0:07:54.320 and that they don't have complete freedom of movement. Most 0:07:54.360 --> 0:07:58.480 of us are not multi jointed in every joint, so 0:07:58.520 --> 0:08:00.600 we can't you know that we have a imitation on 0:08:00.640 --> 0:08:02.920 how far we can move in certain directions with these 0:08:02.960 --> 0:08:06.200 various joints. So taking that into account, you didn't have 0:08:06.280 --> 0:08:10.000 to have sensors all over the body. You would just 0:08:10.040 --> 0:08:12.080 have in a few places, which was good considering that 0:08:12.120 --> 0:08:15.880 there were these thick cables attached to the sensors. And 0:08:15.920 --> 0:08:19.400 then once you were done moving, then the all that 0:08:19.480 --> 0:08:22.920 data would get be captured within the system and could 0:08:22.920 --> 0:08:26.640 then be rendered into animation. Although this was also a 0:08:26.640 --> 0:08:30.640 way that you could do real time animation or digital puppetry. Uh, 0:08:30.640 --> 0:08:33.920 it's not that different from controlling a video game character 0:08:34.360 --> 0:08:37.040 with a controller. It's sort of the same principle, except 0:08:37.080 --> 0:08:39.559 in this case the the video game controller instead of 0:08:39.720 --> 0:08:41.960 being something you hold in your hands, it's something you 0:08:42.000 --> 0:08:46.479 were actually wearing. And uh, I've seen plenty of instances 0:08:46.480 --> 0:08:49.599 of this. If you've ever seen Turtle Talk with Crush 0:08:49.800 --> 0:08:51.800 over at Disney, that's what they use. They use a 0:08:51.840 --> 0:08:55.480 digital you know, they use digital poetry, and it's awesome, 0:08:55.480 --> 0:08:59.760 by the way, I love that. Well, it it would 0:08:59.800 --> 0:09:03.760 all so seeing that, um, you would need to be 0:09:03.800 --> 0:09:07.120 aware of where those cables were going, and it would 0:09:07.160 --> 0:09:09.400 it would also affect the way that you would move. 0:09:09.400 --> 0:09:12.080 You wouldn't move as naturally if you were wearing something 0:09:12.160 --> 0:09:15.920 like that as if you were, you know, unencumbered by 0:09:15.960 --> 0:09:19.120 by that, which, um sort of I think would wind 0:09:19.160 --> 0:09:23.160 itself to an upgrade, which is I think why they 0:09:23.280 --> 0:09:28.080 were so keen on optical system Well, it's also also 0:09:28.360 --> 0:09:30.520 that's very true. It did limit what you could do, 0:09:30.600 --> 0:09:32.320 it could live, It would limit your movement. I mean, 0:09:32.320 --> 0:09:34.080 when you've got these big cables attached to you, you you 0:09:34.200 --> 0:09:37.760 obviously you can't just move freely within a space. Um. 0:09:38.040 --> 0:09:41.240 So it did put some limitations on you. Their limitations 0:09:41.240 --> 0:09:43.319 to the optical systems too, but we'll get into that. 0:09:43.880 --> 0:09:48.920 The The other problem was that the sampling rate for 0:09:49.040 --> 0:09:52.000 the magnetic systems was not as high as it is 0:09:52.040 --> 0:09:54.400 for optical systems. And by sampling rate, what I mean 0:09:54.520 --> 0:09:57.880 is that this the entire system as a whole is 0:09:57.920 --> 0:10:02.760 taking little measure sure mints of from the sensors of 0:10:02.960 --> 0:10:05.800 you know, the orientation of those sensors within the space, 0:10:06.559 --> 0:10:10.840 and it does that several times every second. But the 0:10:10.880 --> 0:10:14.400 sample rate of the magnetic motion capture systems was much 0:10:14.559 --> 0:10:18.720 lower than what it was for than what it would 0:10:18.720 --> 0:10:20.600 be if you were to use an optical system. So 0:10:20.720 --> 0:10:24.480 you're not getting data as frequently, I mean still several 0:10:24.480 --> 0:10:27.560 times a second, but it's not as precise as the 0:10:27.600 --> 0:10:30.560 optical system. So not only were you limited in the 0:10:30.640 --> 0:10:33.480 kind of movements you can make because you had these 0:10:33.679 --> 0:10:38.800 major cables attached to you, but also you couldn't get 0:10:39.280 --> 0:10:44.800 really minute precise measurements on every kind of movement. So 0:10:44.880 --> 0:10:48.040 it wasn't good for things like sports. So you know, 0:10:48.280 --> 0:10:50.800 something like throwing a pitch in baseball, there are a 0:10:50.800 --> 0:10:53.880 lot of movements, a little tiny motions that are involved 0:10:53.880 --> 0:10:57.880 in that. I mean, anyone who's watched slow motion footage 0:10:58.040 --> 0:11:02.280 of a professional baseball pitcher throwing a pitch, you can 0:11:02.320 --> 0:11:06.559 see that there are some incredibly subtle movements that are 0:11:06.559 --> 0:11:09.600 involved in that. And uh, and it takes place over 0:11:09.640 --> 0:11:12.080 a very short period of time. I mean, it's a 0:11:12.200 --> 0:11:17.160 very fast thing to to to measure. Using the magnetic 0:11:17.240 --> 0:11:21.480 motion capture system, you would probably one slow the person 0:11:21.520 --> 0:11:24.200 down because they have all these cables attached to them, 0:11:24.240 --> 0:11:27.760 and to not get enough data to give an accurate 0:11:27.800 --> 0:11:31.200 representation of what had happened in the virtual format. So 0:11:31.200 --> 0:11:34.880 if you were to say, create a video game, a 0:11:34.880 --> 0:11:40.920 baseball video game, the picture would not necessarily behave properly 0:11:41.040 --> 0:11:44.040 if all you did was directly port the data you 0:11:44.120 --> 0:11:47.440 got from the motion capture into the game. Yeah. Another 0:11:47.520 --> 0:11:50.560 drawback of the mechanical systems like that too, is that 0:11:50.559 --> 0:11:53.520 that it's UM. It's the kind of system that not 0:11:53.600 --> 0:11:56.760 only is cumbersome and inaccurate, but it has to be 0:11:56.800 --> 0:12:01.240 calibrated fairly frequently. UM. And you know, there there's there's 0:12:01.280 --> 0:12:03.600 some work that you can do with this. But the 0:12:03.600 --> 0:12:07.840 optical systems that they began to introduce UM, you know, 0:12:07.920 --> 0:12:13.040 generally became an upgrade UM. The only there is one 0:12:13.200 --> 0:12:17.520 big advantage that the mechanical systems do have, though, and 0:12:17.559 --> 0:12:22.000 that is that light. The lighting will not necessarily interfere 0:12:22.040 --> 0:12:24.679 with the different points of motion that are captured by 0:12:24.720 --> 0:12:27.880 the mechanical system UM. And that can be an issue 0:12:28.160 --> 0:12:31.880 with the optical systems UM. You know, because that's that's 0:12:31.920 --> 0:12:34.720 why UM. They will be wearing the people, the actors 0:12:34.720 --> 0:12:38.280 who will be UM having their motions captured by the 0:12:38.360 --> 0:12:41.719 system will be wearing you know, those bright dots so 0:12:41.800 --> 0:12:44.200 that the computer can pick up on that. And at 0:12:44.240 --> 0:12:46.200 the beginning, and these are these early systems, there were 0:12:46.200 --> 0:12:50.360 only so many points action points that they could capture. Um. 0:12:50.400 --> 0:12:53.000 They were very limited in what they could do at first, 0:12:53.160 --> 0:12:55.520 but still you know, somewhat of an upgrade over the 0:12:55.520 --> 0:13:00.080 mechanical Yeah. It also limited what you could have in 0:13:00.120 --> 0:13:03.680 the background, obviously, because you could not have anything that 0:13:03.840 --> 0:13:07.640 was going to be of a similar shade UH. You 0:13:07.640 --> 0:13:11.640 know usually where you're talking about reflective white substance used 0:13:11.679 --> 0:13:16.959 as the um the points of of UH articulations. So 0:13:17.000 --> 0:13:21.520 the little like white stickers is like what you were saying, Chris, Um, 0:13:21.600 --> 0:13:23.640 you couldn't have anything like that in the background because 0:13:23.640 --> 0:13:28.040 it would confuse the optical system. So that's why a 0:13:28.040 --> 0:13:31.560 lot of these motion capture scenes are shot against a 0:13:31.559 --> 0:13:35.440 blue screen or green screen. It's so that the background 0:13:35.520 --> 0:13:39.240 does not in any way interfere with the motion capture. 0:13:39.320 --> 0:13:41.600 So if you've ever seen behind the scenes footage of 0:13:41.720 --> 0:13:43.520 The Lord of the Rings movies is a great example 0:13:44.040 --> 0:13:48.280 with Andy Sarkis as Gollum or Sniegel if you prefer, 0:13:48.760 --> 0:13:53.040 But he's wearing you know, a tight like skin tight 0:13:53.120 --> 0:13:57.400 suit with these little white UH circles all over it. 0:13:57.720 --> 0:14:01.600 Those are the points that the camera track to create 0:14:01.720 --> 0:14:06.800 the the performance of gallam slash Sniegel. So the performance 0:14:06.880 --> 0:14:10.920 is something that's being created not only by the actor 0:14:11.120 --> 0:14:13.920 but also the animators because not we should also point 0:14:13.960 --> 0:14:17.840 out that the motion capture stuff rarely is motion capture 0:14:18.760 --> 0:14:24.080 uh completely. Uh there's there's rarely a moment where you 0:14:24.120 --> 0:14:26.800 don't have an animator step in and tweak it somehow, 0:14:27.360 --> 0:14:32.400 Like you don't normally have someone create a physical performance 0:14:32.520 --> 0:14:39.160 and that physical performance is completely without any tinkering represented 0:14:39.160 --> 0:14:41.440 in the final product. I mean it can happen, there 0:14:41.480 --> 0:14:44.760 are instances of it, but it's more frequently, uh, something 0:14:44.800 --> 0:14:48.280 where the motion capture performance goes to the animator who 0:14:48.320 --> 0:14:52.040 can then tweak things if the performance is not exactly 0:14:52.040 --> 0:14:54.960 what needs to be, which is kind of nice. You 0:14:54.960 --> 0:14:58.120 don't necessarily have that luxury with flesh and blood actors. 0:14:58.560 --> 0:15:02.040 I'm gonna stop motion ure in mid motion right now 0:15:02.320 --> 0:15:04.000 so that we can take a quick break to thank 0:15:04.040 --> 0:15:15.360 our sponsor. Well, especially with the earlier systems, especially the 0:15:15.760 --> 0:15:21.480 electromagnetic systems, Uh, those were really noisy, not literally noisy, 0:15:21.520 --> 0:15:25.880 but digital noise. They weren't they weren't really highly accurate. Um. 0:15:25.920 --> 0:15:29.440 The optical systems are are far cleaner and give them 0:15:29.440 --> 0:15:32.520 more accurate representation. But you know that that's it sort 0:15:32.560 --> 0:15:35.920 of falls in the realm of artistic license. I would think, um, 0:15:35.960 --> 0:15:38.840 where they need to go in and make subtle adjustments 0:15:38.880 --> 0:15:41.840 to make it look the way they wanted to look. Ye. 0:15:42.360 --> 0:15:44.760 I should also point out, now you just reminded me 0:15:44.760 --> 0:15:47.880 of something else another drawback to the electromagnetic systems, which 0:15:47.960 --> 0:15:52.640 was you couldn't have anything metal on the set because 0:15:52.720 --> 0:15:56.000 it would interfere with that magnetic field and give incorrect 0:15:56.040 --> 0:15:59.600 readings to the system. So you're you're virtual character would 0:15:59.640 --> 0:16:01.720 not move in the same way as the physical one 0:16:01.760 --> 0:16:04.240 because there would be some interference in that sense. So 0:16:04.560 --> 0:16:08.600 your set couldn't have anything metal in it. The props 0:16:08.880 --> 0:16:11.840 didn't shouldn't have anything metal in them, so that that 0:16:11.960 --> 0:16:14.600 limited to you as well. So each system has its 0:16:14.600 --> 0:16:18.520 own limitations. Getting back to the optical one, UM, one 0:16:18.520 --> 0:16:20.080 of the other things you have to remember is that 0:16:20.120 --> 0:16:24.920 in order to really capture a a physical object moving 0:16:24.960 --> 0:16:30.840 through three D space and to replicate that in virtual space, 0:16:31.600 --> 0:16:35.760 you need multiple cameras in that system because a single camera, 0:16:36.120 --> 0:16:40.280 assuming that's a regular video or film camera, something that 0:16:40.360 --> 0:16:44.520 does not have three D capability, pointing that at an 0:16:44.520 --> 0:16:48.840 object it's creating a two dimensional image of something that's 0:16:48.920 --> 0:16:54.560 moving in three dimensions. The camera can't necessarily tell where 0:16:54.880 --> 0:16:59.880 movements are happening within the depth frame of of that 0:17:00.120 --> 0:17:02.840 of that image, right, So if someone's moving in such 0:17:02.880 --> 0:17:05.240 a way where let's say they're moving their head where 0:17:05.240 --> 0:17:09.800 it would be bobbing closer to the camera. Uh, unless 0:17:10.000 --> 0:17:15.320 the size of the the sensors is such that something 0:17:15.359 --> 0:17:17.960 that's subtle could be picked up by the camera system, 0:17:18.000 --> 0:17:21.399 you would lose that information. So what you need are 0:17:21.480 --> 0:17:25.120 multiple cameras on the same object so that you can 0:17:25.200 --> 0:17:28.800 compare that data from the multiple angles to tell how 0:17:28.840 --> 0:17:32.080 this object is really moving through this three dimensional space. 0:17:32.560 --> 0:17:35.800 So it's kind of like the idea of having parallax 0:17:36.040 --> 0:17:39.280 with two eyes. You know, our eyes are offset, so 0:17:39.440 --> 0:17:42.640 by looking at an object, we can tell how far 0:17:42.680 --> 0:17:47.239 away it is in part because of parallax. Uh. We 0:17:47.320 --> 0:17:49.840 also have other visual cues that tell us about how 0:17:49.880 --> 0:17:52.439 far something is, you know, things like how tall it 0:17:52.560 --> 0:17:54.600 is in relation to where we are that kind of thing, 0:17:54.720 --> 0:17:56.520 or how tall it is in relation to other objects 0:17:56.560 --> 0:17:59.520 that are within our frame of vision. But parallax is 0:17:59.600 --> 0:18:02.359 very importan and same sort of thing with these optical systems, 0:18:02.359 --> 0:18:04.800 you would have multiple cameras set up to try and 0:18:04.920 --> 0:18:09.840 capture the information that's going on in the frame so 0:18:09.920 --> 0:18:12.560 that you could tell exactly how it's moving through that 0:18:12.600 --> 0:18:16.800 three dimensional space. Yeah, it seems like um, in order 0:18:16.840 --> 0:18:20.760 to capture the correct perspective, you need that additional information, 0:18:21.240 --> 0:18:24.080 even though you may not necessarily see it. UM. It 0:18:24.119 --> 0:18:27.080 helps the the animator do that, and the optical system 0:18:27.200 --> 0:18:31.879 to allows you to work with more than one actor um, 0:18:31.920 --> 0:18:34.800 which was not really an option with some of the 0:18:34.800 --> 0:18:39.359 earlier systems. So in other words, you can although it 0:18:39.400 --> 0:18:43.040 requires more equipment, you know, just simply out of necessity, 0:18:43.160 --> 0:18:47.960 the optical system is really affording the animators a an 0:18:48.000 --> 0:18:52.720 opportunity to use a greater amount of information um both 0:18:52.920 --> 0:18:56.320 you know, from the different the different points of data 0:18:56.359 --> 0:18:59.760 they're getting from a single actor, but from multiple actors 0:18:59.760 --> 0:19:03.160 on this set simultaneously, which enables them to to create 0:19:03.160 --> 0:19:08.119 more complex work. Right and uh. This also gives us 0:19:08.280 --> 0:19:11.440 a good example of how the optical motion capture systems 0:19:11.440 --> 0:19:15.320 are a passive system because you have these sensors you're 0:19:15.359 --> 0:19:18.359 wearing that are not necessarily or not even sensors, they're 0:19:18.480 --> 0:19:21.679 they're reflective markers that you're wearing. They aren't connected to 0:19:21.720 --> 0:19:26.240 any sort of electronic components at all, versus the active 0:19:26.440 --> 0:19:32.160 systems like the electromagnetic one where you are generating data 0:19:32.240 --> 0:19:34.840 by moving through a magnetic field and you have these 0:19:34.880 --> 0:19:38.280 big cables attached to it. Uh with the optical motion 0:19:38.280 --> 0:19:40.479 capture systems. And another thing that's kind of interesting I 0:19:40.520 --> 0:19:42.680 think is that a lot of at least the early ones, 0:19:43.119 --> 0:19:47.560 the cameras would have infrared l e ed s uh 0:19:47.600 --> 0:19:51.640 so emitters. Really that we're emitting infrared lights that's outside 0:19:51.720 --> 0:19:55.919 are our visible spectrum. We cannot see infrared light. But 0:19:56.520 --> 0:19:59.480 by putting an infrared filter on the camera, you could 0:19:59.640 --> 0:20:03.320 have the camera pick up reflections of infrared light. And 0:20:03.400 --> 0:20:07.360 that was a way of helping to identify the sensors 0:20:07.400 --> 0:20:10.199 that you had put on the actor. The actors, the 0:20:10.280 --> 0:20:14.400 sensors would be reflective specifically so that the infrared light 0:20:14.440 --> 0:20:17.280 would reflect back toward the camera and give the most 0:20:17.320 --> 0:20:21.119 accurate rendering of what's going on at any given moment 0:20:21.240 --> 0:20:25.800 within a scene. So, um, yeah, it's another way of 0:20:25.840 --> 0:20:29.760 making sure that the data being captured is as precise 0:20:29.800 --> 0:20:31.480 as possible. I mean, that is, of course, the goal 0:20:31.560 --> 0:20:36.160 is to try and recreate the physical movements as truthfully 0:20:36.200 --> 0:20:41.200 as you possibly can given all the limitations involved. Yeah, 0:20:41.240 --> 0:20:44.359 and if you're looking for a real life easy to 0:20:44.520 --> 0:20:48.800 find an example of this, you would look no farther 0:20:48.920 --> 0:20:53.880 than your local video game store. Um, because the Xbox 0:20:53.960 --> 0:20:59.840 connect uses very much that that exact form of technology 0:21:00.119 --> 0:21:03.280 is using an infrared emitter UM, and it has cameras 0:21:03.280 --> 0:21:06.000 that it uses to pick it up uh, the information, 0:21:06.240 --> 0:21:08.800 pick the information up that is coming back from what 0:21:08.960 --> 0:21:11.520 is being reflected around the room. And anybody who who 0:21:11.520 --> 0:21:14.880 has one is also aware that lighting is very much 0:21:14.880 --> 0:21:17.800 an issue. UM. The way that were room is let 0:21:17.920 --> 0:21:20.920 affects the information that the connect is able to refer 0:21:21.000 --> 0:21:24.000 to the Xbox. Now, it's not, while it is sophisticated, 0:21:24.080 --> 0:21:26.199 is not as sophisticated as the kind of equipment that 0:21:26.240 --> 0:21:28.919 they might use in making a movie or making a 0:21:29.080 --> 0:21:32.800 video game. But it is very very similar technology, and 0:21:32.800 --> 0:21:35.760 in some ways I would argue that it's more sophisticated 0:21:35.760 --> 0:21:38.840 than some of those early UH systems simply because it 0:21:38.960 --> 0:21:42.600 is able to capture a lot of information. UM. Whereas 0:21:42.680 --> 0:21:45.359 you know, the very early optical systems were only using 0:21:45.560 --> 0:21:49.240 a handful of data points, right, So UM, it's it's 0:21:50.200 --> 0:21:53.200 a pretty neat device. Um, you know, not only used 0:21:53.200 --> 0:21:55.240 for gaming. Now the hacker community has fallen in love 0:21:55.280 --> 0:21:57.800 with it too because it can do so much and 0:21:57.800 --> 0:21:59.879 can be used for so many things and is you know, 0:22:00.000 --> 0:22:03.119 fairly inexpensive. Yeah. The cool thing about the Connect is 0:22:03.160 --> 0:22:06.040 that rather than have to obviously, if you've if you've 0:22:06.080 --> 0:22:09.000 ever played an Xbox with a connect, you know you 0:22:09.040 --> 0:22:12.040 don't have to go out and buy a snug body 0:22:12.080 --> 0:22:14.879 suit covered in reflective markers in order to play. I mean, 0:22:14.920 --> 0:22:18.119 it doesn't hurt, but you know, if you're if you 0:22:18.119 --> 0:22:20.240 can pull that look off. There are very few of 0:22:20.280 --> 0:22:23.800 us who can. I count myself among them. But you 0:22:23.840 --> 0:22:25.800 don't have to do that because what it's doing is 0:22:25.800 --> 0:22:30.880 it's actually projecting essentially a grid uh in infrared light, 0:22:31.240 --> 0:22:33.680 so you can't see the grid, but it's being projected 0:22:33.960 --> 0:22:37.720 into the room, and then when you move uh within 0:22:37.760 --> 0:22:40.320 the space, you are deforming that grid. You know, the 0:22:40.359 --> 0:22:45.120 camera that's picking up the reflections of that infrared light 0:22:45.760 --> 0:22:48.920 can detect when the grid's being deformed by a physical 0:22:49.000 --> 0:22:52.680 object interrupting the grid. So as you move, you interrupt 0:22:52.680 --> 0:22:54.480 different parts of the grid, and it can start to 0:22:54.640 --> 0:23:00.840 interpret those as motions and commands. It's not uh, it's 0:23:00.920 --> 0:23:03.600 not as precise as what we're talking about with the 0:23:03.640 --> 0:23:06.959 optical systems that are used in movies and video games. Uh, 0:23:07.040 --> 0:23:11.000 to to create them, that is, not to to play them. Um, 0:23:11.000 --> 0:23:13.960 it's not as precise as those. But it also has 0:23:14.080 --> 0:23:17.280 other elements that help balance it out, Like it has 0:23:18.000 --> 0:23:23.920 regular optical cameras that can have some other software that 0:23:24.280 --> 0:23:28.400 aids it in recognizing things like facial recognition software, which 0:23:28.440 --> 0:23:32.760 does not necessarily rely upon that infrared grid. It relies 0:23:32.800 --> 0:23:37.440 more on the traditional camera functions, but has the software 0:23:37.520 --> 0:23:42.040 included that lets the the programs within recognize who is 0:23:42.080 --> 0:23:46.640 standing in front of it, so that combination increases the precision, 0:23:46.640 --> 0:23:48.920 which of course is very important whenever you're playing a game. 0:23:48.960 --> 0:23:51.560 I mean, anyone who's played any sort of game where 0:23:51.600 --> 0:23:55.000 you're using a faulty controller, or it's just a system 0:23:55.080 --> 0:23:59.120 that hasn't been fully uh it's not finished yet, it's 0:23:59.119 --> 0:24:01.679 just in prototypes, dage or whatever. You may have noticed 0:24:01.720 --> 0:24:05.720 that it could be very frustrating to try and control 0:24:05.880 --> 0:24:10.240 something where the actual controller is not as responsive as 0:24:10.280 --> 0:24:15.200 you would hope. It's um not a fun experience, but anyway, 0:24:15.280 --> 0:24:20.920 that is kind of related to this whole motion capture technology. UM. 0:24:22.200 --> 0:24:24.000 I'm sorry what you were You look like you have 0:24:24.040 --> 0:24:26.400 something to say, Well, no, I was, I was going 0:24:26.480 --> 0:24:29.680 to say that. UM. You know, we really hadn't other 0:24:29.720 --> 0:24:33.440 than my earlier statement about sports. UM. You know, we've 0:24:33.440 --> 0:24:36.680 we've been talking about it in an entertain amount entertainment 0:24:37.200 --> 0:24:42.080 about the the ability to capture motion to make characters 0:24:42.119 --> 0:24:46.640