WEBVTT - How Motion Capture Works 0:00:00.520 --> 0:00:03.160 Brought to you by the two thousand and twelve Toyota Camera. 0:00:05.880 --> 0:00:08.920 Get in touch with technology with tech Stuff from how 0:00:08.960 --> 0:00:17.000 stuff works dot com. Hello again, everyone, and welcome to 0:00:17.040 --> 0:00:19.000 tech stuff. My name is Chris Poulette, and I'm an 0:00:19.079 --> 0:00:21.720 editor at how stuff works dot com. Sitting across from 0:00:21.720 --> 0:00:24.440 me as always as senior writer Jonathan Strickland. Hey there, 0:00:25.200 --> 0:00:28.640 So today we thought we'd talk a bit about a 0:00:28.760 --> 0:00:33.720 type of performance that is relatively new as far as 0:00:33.760 --> 0:00:37.680 performance goes. Uh, something that uh, I guess this falls 0:00:37.680 --> 0:00:41.080 into our movie making category, but it's also something that's 0:00:41.120 --> 0:00:44.120 been used in things like video games and and other 0:00:44.159 --> 0:00:48.640 forms of media as well. Motion capture. Yeah, you'll even 0:00:48.680 --> 0:00:52.720 see it in uh, in sports, they've been talking about 0:00:52.720 --> 0:00:55.120 this for a while now. And if you've ever seen 0:00:55.480 --> 0:01:01.040 the making of a a video or a game, um, 0:01:01.160 --> 0:01:05.400 or you know, even in sports rehabilitation in medicine, um, 0:01:05.440 --> 0:01:08.480 they where the people are wearing dots, little white dots 0:01:08.520 --> 0:01:12.880 all over their clothing and sometimes their faces and hands. UM, 0:01:12.920 --> 0:01:15.679 that's probably what they were doing. Either that or they 0:01:15.760 --> 0:01:18.960 just really like stickers. Yeah, yeah, I mean, who doesn't. 0:01:19.120 --> 0:01:22.880 I remember being very competitive in elementary school in order 0:01:22.920 --> 0:01:25.800 to get a sticker. And also this is a tangent 0:01:25.880 --> 0:01:29.120 but a true story. I got a gold star sticker 0:01:29.520 --> 0:01:32.920 uh just last month awes from Tracy, the head of 0:01:32.920 --> 0:01:37.959 our our site. So anyway, um, yeah, motion capture. Actually, 0:01:38.000 --> 0:01:40.759 there are a lot of different terms that you can 0:01:40.920 --> 0:01:44.800 use in this in this realm Uh, motion capture or 0:01:44.840 --> 0:01:47.600 mo cap is probably the one I hear the most frequently, 0:01:47.640 --> 0:01:53.120 but also things like performance animation, performance capture, digital puppetry, 0:01:53.280 --> 0:01:57.080 real time animation, motion scanning, which is really more of 0:01:57.080 --> 0:02:01.000 a proprietary thing, but these are The concept is pretty 0:02:01.040 --> 0:02:03.000 much the same across the board. The idea is to 0:02:03.240 --> 0:02:08.919 capture the physical representation of something and then converted into 0:02:09.000 --> 0:02:12.799 a virtual format. So usually it's something that's in motion, 0:02:12.840 --> 0:02:15.560 but it's not always that way. Uh, since you know 0:02:15.600 --> 0:02:19.280 we're talking about motion capture, that makes sense. But you're 0:02:19.320 --> 0:02:23.359 trying to get uh, translate something that is moving through 0:02:23.440 --> 0:02:28.560 real space into a digital format. And uh, there's different 0:02:28.600 --> 0:02:30.000 ways to do this. I mean, you could do it 0:02:30.080 --> 0:02:32.639 the really hard way, which is where you study something 0:02:32.680 --> 0:02:35.680 and then you try to recreate it, uh, either by 0:02:35.760 --> 0:02:38.560 hand or by or digitally, you know, by by programming 0:02:39.080 --> 0:02:43.400 movements into an animated figure. But this is an idea 0:02:43.400 --> 0:02:46.000 that kind of takes that step out where you are 0:02:46.560 --> 0:02:50.800 directly porting the movements. Uh, something is making within physical 0:02:50.800 --> 0:02:55.200 space into virtual space. Yeah, there was an early technique, um. 0:02:55.240 --> 0:02:56.960 And of course this is this is all an attempt 0:02:57.040 --> 0:03:01.720 to get as real as you can with of animation. UM. 0:03:02.400 --> 0:03:06.240 And one of the earlier techniques that that was sort 0:03:06.280 --> 0:03:10.359 of a predecessor to this is called rotoscoping. Uh. Ralph 0:03:10.400 --> 0:03:13.400 Box She's Lord of the Rings had a lot of 0:03:13.520 --> 0:03:17.520 rotoscoping in it. Well, what happens is, um, in that 0:03:17.600 --> 0:03:21.440 case is that a real uh, real human being goes 0:03:21.520 --> 0:03:24.480 through the motions and they act through the parts that 0:03:24.520 --> 0:03:26.760 are that you're going to see in the animation. They 0:03:26.760 --> 0:03:30.600 shoot that on film, yes, yes, and then the the 0:03:30.639 --> 0:03:34.640 animators basically are looking at that and are drawing more 0:03:34.720 --> 0:03:37.160 or less on top of that. They see a projection 0:03:37.160 --> 0:03:39.800 of that, and they are drawing, uh, the animation over 0:03:39.840 --> 0:03:44.080 that to capture the way that person's body looks. And 0:03:44.880 --> 0:03:47.760 this this was famous. You know, the Disney studios were 0:03:47.760 --> 0:03:50.120 famous for this. We're studying models and then they would 0:03:50.120 --> 0:03:54.600 do the rotoscoping technique to to try to make their uh, 0:03:55.000 --> 0:03:58.480 their characters look more realistic. Yeah. And there are some artists, 0:03:58.560 --> 0:04:02.480 like I said, like box She who famously would leave 0:04:02.560 --> 0:04:06.120 the film image as part of the animation, so that 0:04:06.160 --> 0:04:09.120 you had this this weird effect where the thing you 0:04:09.160 --> 0:04:13.360 were looking at was part uh well quote unquote real 0:04:13.400 --> 0:04:17.240 image and part animated image, which was it was an 0:04:17.320 --> 0:04:21.200 artistic choice, uh, definitely something that was not meant to 0:04:21.200 --> 0:04:24.280 to necessarily fool you into thinking, oh, well, that animated 0:04:24.360 --> 0:04:26.960 character is moving very realistically. It was done on purpose, 0:04:27.560 --> 0:04:29.560 but it was. That's what I always think of when 0:04:29.600 --> 0:04:32.119 I think rhodoscoping, as I just think of the different 0:04:32.200 --> 0:04:34.040 box Sheet films, but in particular I think of his 0:04:34.160 --> 0:04:37.640 Lord of the Rings adaptation um, which, as I recall, 0:04:37.800 --> 0:04:42.800 ended halfway through the Two Towers. So anyway, that's just 0:04:42.839 --> 0:04:45.240 bringing back memories. But yeah, that was that was sort 0:04:45.240 --> 0:04:50.560 of a precursor to motion capture. Motion capture itself. There 0:04:50.560 --> 0:04:55.599 are many different ways of achieving this. For example, there 0:04:55.680 --> 0:05:00.159 were it's not used very frequently now, but there were 0:05:00.800 --> 0:05:04.960 mechanical systems where you had sensors that would be attached 0:05:05.000 --> 0:05:10.640 to specific joints uh that would relay movement. And usually 0:05:10.640 --> 0:05:12.839 it was kind of like a like an actor would 0:05:12.839 --> 0:05:19.039 wear a physical metallic skeleton type device that would have 0:05:19.160 --> 0:05:22.800 the sensors attached to the various joints and as the 0:05:22.839 --> 0:05:27.719 actor moved, the sensors would register the changes in motion 0:05:28.600 --> 0:05:33.160 in this metallic skeleton and UH, and that would be 0:05:33.200 --> 0:05:38.560 relayed through usually cables to a computer system that would 0:05:39.120 --> 0:05:42.080 measure these or take the measurements from the sensors and 0:05:42.080 --> 0:05:45.960 translated into movements for the virtual character. UH. It's very 0:05:46.040 --> 0:05:49.839 limiting this particular system. There was another one that was 0:05:49.920 --> 0:05:55.280 a little more versatile, which was used electro magnets. And 0:05:55.320 --> 0:05:58.640 in this case you talked about sensors that would be 0:05:58.680 --> 0:06:02.279 attached by really thin cables that again would go to 0:06:02.520 --> 0:06:06.080 a computer, and there'd be a magnetic field and by 0:06:06.160 --> 0:06:10.440 moving through this magnetic field, the sensors would pick up alterations. 0:06:10.440 --> 0:06:13.000 You know, it would you know, moving through magnetic field, 0:06:13.000 --> 0:06:16.880 you would get little electrical changes. We we've talked a 0:06:16.880 --> 0:06:22.880 lot about electricity magnetism in general, moving through UH, Fluctuating 0:06:22.880 --> 0:06:27.039 a magnetic field can induce electricity through a conductor, or 0:06:27.400 --> 0:06:30.800 putting electricity through a conductor can induce a magnetic field. 0:06:31.000 --> 0:06:34.240 So anyway, by moving these sensors through the magnetic field, 0:06:34.440 --> 0:06:38.359 it would create these electronic fluctuations that would then be 0:06:38.480 --> 0:06:43.359 measured and translated into movement, and again this was a 0:06:43.560 --> 0:06:47.880 fairly effective way of picking up movements. It actually didn't 0:06:48.000 --> 0:06:50.960 use as many points of contact as the optical systems 0:06:50.960 --> 0:06:52.919 that we mostly think about. That was the kind that 0:06:53.000 --> 0:06:55.599 Chris was referring to early with all the dots on 0:06:55.640 --> 0:06:58.960 the person. Those systems tend to have lots and lots 0:06:58.960 --> 0:07:02.800 and lots of points of ref. The electro magnet ones 0:07:03.279 --> 0:07:06.799 didn't tend to have as many points of reference because 0:07:07.320 --> 0:07:10.640 the the software side of it, because you know, we 0:07:10.680 --> 0:07:12.520 do have a hardware and a software side to this. 0:07:12.880 --> 0:07:16.360 The software side would assume that the joints that these 0:07:16.400 --> 0:07:20.440 sensors were attached to behaved the way they normally would 0:07:20.480 --> 0:07:24.239 in humans, and that they don't have complete freedom of movement. 0:07:24.440 --> 0:07:28.600 Most of us are not multi jointed in every joint, 0:07:28.720 --> 0:07:30.800 so we can't you know, that we have a limitation 0:07:30.840 --> 0:07:33.080 on how far we can move in certain directions with 0:07:33.120 --> 0:07:36.440 these various joints. So taking that into account, you didn't 0:07:36.480 --> 0:07:40.040 have to have sensors all over the body. You would 0:07:40.240 --> 0:07:41.920 just have them in a few places, which was good 0:07:41.960 --> 0:07:44.520 considering that there were these thick cables attached to the 0:07:44.600 --> 0:07:49.080 sensors and then once you were done moving, then the 0:07:49.480 --> 0:07:52.720 all that data would get be captured within the system 0:07:52.800 --> 0:07:56.600 and could then be rendered into animation. Although this was 0:07:56.640 --> 0:07:59.200 also a way that you could do real time animation 0:07:59.320 --> 0:08:02.720 or digital a tree. Uh, it's not that different from 0:08:02.720 --> 0:08:06.160 controlling a video game character with a controller. It's sort 0:08:06.160 --> 0:08:08.320 of the same principle, except in this case the the 0:08:08.560 --> 0:08:10.800 video game controller. Instead of it being something you hold 0:08:10.800 --> 0:08:15.120 in your hands, it's something you were actually wearing. And uh, 0:08:15.240 --> 0:08:17.600 I've seen plenty of instances of this. If you've ever 0:08:17.600 --> 0:08:21.400 seen Turtle Talk with Crush over at Disney, that's what 0:08:21.480 --> 0:08:23.280 they use. They use a digital you know, they use 0:08:23.320 --> 0:08:27.440 digital poetry, and it's awesome by the way, I love that. 0:08:28.360 --> 0:08:32.200 Well it uh, it would also seem that, um, you 0:08:32.200 --> 0:08:36.559 would need to be aware of where those cables were going, 0:08:37.080 --> 0:08:39.280 and it would it would also affect the way that 0:08:39.280 --> 0:08:41.600 you would move. You wouldn't move as naturally if you 0:08:41.640 --> 0:08:44.360 were wearing something like that as if you were, you know, 0:08:44.760 --> 0:08:48.920 unencumbered by by that, which, um, sort of I think 0:08:49.000 --> 0:08:52.400 would lend itself to to an upgrade, which is I 0:08:52.440 --> 0:08:57.839 think why they were so keen on Well, it's also 0:08:58.120 --> 0:09:00.920 also that's very true. It did limit what you could do, 0:09:01.000 --> 0:09:02.720 it could live, It would limit your movement. I mean, 0:09:02.800 --> 0:09:05.000 we've got these big cables attached to you. You You obviously 0:09:05.040 --> 0:09:08.559 you can't just move freely within a space. Um, So 0:09:08.640 --> 0:09:11.679 it did put some limitations on you. Their limitations to 0:09:11.720 --> 0:09:15.560 the optical systems too, but we'll get into that. The 0:09:14.800 --> 0:09:19.320 uh the other problem was that the sampling rate for 0:09:19.400 --> 0:09:22.360 the magnetic systems was not as high as it is 0:09:22.400 --> 0:09:24.800 for optical systems. And by sampling rate, what I mean 0:09:24.880 --> 0:09:28.240 is that this the entire system as a whole, is 0:09:28.280 --> 0:09:33.520 taking little measurements of from the sensors of you know, 0:09:33.559 --> 0:09:37.200 the orientation of those sensors within the space, and it 0:09:37.280 --> 0:09:42.000 does that several times every second. But the sample rate 0:09:42.080 --> 0:09:46.160 of the magnetic motion capture systems was much lower than 0:09:46.240 --> 0:09:49.400 what it was for than what it would be if 0:09:49.400 --> 0:09:51.480 you were to use an optical system. So you're not 0:09:51.559 --> 0:09:55.520 getting data as frequently. I mean still several times a second, 0:09:55.559 --> 0:09:59.000 but it's not as precise as the optical system. So 0:09:59.520 --> 0:10:01.720 not only where you limited in the kind of movements 0:10:01.760 --> 0:10:05.640 you can make because you had these major cables attached 0:10:05.679 --> 0:10:11.520 to you, but also you couldn't get really minute precise 0:10:12.480 --> 0:10:16.040 measurements on every kind of movement, So it wasn't good 0:10:16.040 --> 0:10:19.480 for things like sports. So, you know, something like throwing 0:10:19.480 --> 0:10:22.200 a pitch in baseball, there are a lot of movements, 0:10:22.200 --> 0:10:25.160 a little tiny motions that are involved in that. I mean, 0:10:25.360 --> 0:10:30.880 anyone who's watched slow motion footage of a professional baseball 0:10:30.920 --> 0:10:33.800 pitcher throwing a pitch, you can see that there are 0:10:33.840 --> 0:10:38.880 some incredibly subtle movements that are involved in that. And uh, 0:10:38.920 --> 0:10:41.960 and it takes place over a very short period of time. 0:10:42.000 --> 0:10:45.600 I mean, it's a very fast thing to to to measure. 0:10:46.600 --> 0:10:50.400 Using the magnetic motion capture system, you would probably one 0:10:51.160 --> 0:10:53.640 slow the person down because they have all these cables 0:10:53.679 --> 0:10:56.440 attached to them, and to not get enough data to 0:10:56.920 --> 0:11:00.520 give an accurate representation of what had happened in the 0:11:00.600 --> 0:11:04.480 virtual format. So if you were to say, create a 0:11:04.559 --> 0:11:07.520 video game, a baseball video game, the picture would not 0:11:07.679 --> 0:11:13.760 necessarily behave properly if all you did was directly port 0:11:13.800 --> 0:11:17.560 the data you got from the motion capture into the game. Yeah. 0:11:17.559 --> 0:11:20.640 Another drawback of the mechanical systems like that too, is 0:11:20.679 --> 0:11:23.760 that that it's um it's the kind of system that 0:11:23.800 --> 0:11:27.040 not only is cumbersome and inaccurate, but it has to 0:11:27.040 --> 0:11:31.200 be calibrated fairly frequently. UM. And you know, there there's 0:11:31.200 --> 0:11:33.640 a there's some work that you can do with this. 0:11:33.760 --> 0:11:37.920 With the optical systems that they began to introduce UM, 0:11:37.960 --> 0:11:43.040 you know, generally became an upgrade UM. The only there 0:11:43.120 --> 0:11:47.760 is one big advantage that the mechanical systems do have, though, 0:11:47.800 --> 0:11:51.400 and that is that light. The lighting will not necessarily 0:11:51.400 --> 0:11:54.880 interfere with the different points of motion that are captured 0:11:54.880 --> 0:11:57.880 by the mechanical system UM. And that can be an 0:11:57.880 --> 0:12:01.960 issue with the optical systems UM. You know, because that's 0:12:02.080 --> 0:12:04.640 that's why UM, they will be wearing the people, the 0:12:04.679 --> 0:12:08.560 actors who will be UM having their motions captured by 0:12:08.600 --> 0:12:11.760 the system, will be wearing you know, those bright dots 0:12:11.920 --> 0:12:14.400 so that the computer can pick up on that. And 0:12:14.440 --> 0:12:16.440 at the beginning, and these are these early systems, there 0:12:16.440 --> 0:12:20.720 were only so many points action points that they could capture. UM. 0:12:20.800 --> 0:12:23.400 They were very limited in what they could do at first, 0:12:23.520 --> 0:12:25.880 but still you know, somewhat of an upgrade over the 0:12:25.920 --> 0:12:30.400 mechanical Yeah. It also limited what you could have in 0:12:30.440 --> 0:12:34.040 the background, obviously, because you could not have anything that 0:12:34.200 --> 0:12:38.079 was going to be of a similar shade. Uh. You know, 0:12:38.160 --> 0:12:42.199 usually where you're talking about reflective white substance used as 0:12:42.240 --> 0:12:47.440 the um the points of of uh articulations. So the 0:12:47.600 --> 0:12:50.600 little like white stickers is like what you were saying, 0:12:50.640 --> 0:12:53.199 Chris Um, you couldn't have anything like that in the 0:12:53.240 --> 0:12:57.840 background because it would confuse the optical system. So that's 0:12:57.880 --> 0:13:01.439 why a lot of these motion capture scenes are shot 0:13:01.440 --> 0:13:04.760 against a blue screen or green screen. It's so that 0:13:05.040 --> 0:13:08.720 the background does not in any way interfere with the 0:13:08.800 --> 0:13:11.240 motion capture. So if you've ever seen behind the scenes 0:13:11.240 --> 0:13:13.280 footage of The Lord of the Rings movies is a 0:13:13.280 --> 0:13:18.120 great example with Andy Serkis as Gollum or Sniegel if 0:13:18.160 --> 0:13:22.480 you prefer, but he's wearing you know, a tight like 0:13:22.760 --> 0:13:27.360 skin tight suit with these little white uh circles all 0:13:27.400 --> 0:13:30.800 over it. Those are the points that the cameras track 0:13:31.280 --> 0:13:36.080 to create the the performance of Gollum slash Sniegel. So 0:13:36.480 --> 0:13:39.880 the performance is something that's being created not only by 0:13:40.120 --> 0:13:43.840 the actor but also the animators because not We should 0:13:43.880 --> 0:13:47.240 also point out that the motion capture stuff rarely is 0:13:47.320 --> 0:13:54.120 motion capture uh completely. Uh. There's there's rarely a moment 0:13:54.120 --> 0:13:56.520 where you don't have an animator step in and tweak 0:13:56.559 --> 0:14:01.520 it somehow, like Uh, you don't normally have someone create 0:14:01.559 --> 0:14:06.840 a physical performance and that physical performance is completely without 0:14:06.920 --> 0:14:10.719 any tinkering represented in the final product. I mean it 0:14:11.040 --> 0:14:14.760 can happen, there are instances of it, but it's more frequently, uh, 0:14:14.800 --> 0:14:18.439 something where the motion capture performance goes to the animator, 0:14:18.480 --> 0:14:21.720 who can then tweak things if the performance is not 0:14:21.840 --> 0:14:24.840 exactly what needs to be, which is kind of nice. 0:14:25.240 --> 0:14:28.480 You don't necessarily have that luxury with flesh and blood actors. 0:14:29.280 --> 0:14:33.000 That's that's true. That's true. Well, especially with the earlier systems, 0:14:33.080 --> 0:14:38.440 especially the electromagnetic systems. Uh, those were really noisy, not 0:14:38.520 --> 0:14:42.120 literally noisy, but but digital noise. They weren't They weren't 0:14:42.160 --> 0:14:46.240 really highly accurate. Um. The optical systems are are far 0:14:46.440 --> 0:14:49.920 cleaner and give them more accurate representation. But you know there, 0:14:50.000 --> 0:14:52.720 that's it's sort of falls in the realm of artistic license. 0:14:52.720 --> 0:14:55.200 I would think, um, where they need to go in 0:14:55.240 --> 0:14:59.000 and make subtle adjustments to make it look the way 0:14:59.000 --> 0:15:01.560 they want it to look. Up. I should also point out, 0:15:01.720 --> 0:15:04.240 now you just reminded me of something else another drawback 0:15:04.280 --> 0:15:07.720 to the electromagnetic systems, which was you couldn't have anything 0:15:07.760 --> 0:15:11.840 metal on the set because it would interfere with that 0:15:11.880 --> 0:15:15.520 magnetic field and give incorrect readings to the system. So 0:15:15.600 --> 0:15:18.880 you're you're virtual character would not move in the same 0:15:18.920 --> 0:15:20.640 way as the physical one because there would be some 0:15:20.680 --> 0:15:24.440 interference in that sense. So your set couldn't have anything 0:15:24.600 --> 0:15:28.640 metal in it. The props didn't shouldn't have anything metal 0:15:28.800 --> 0:15:31.160 in them, so that that limited to you as well. 0:15:31.240 --> 0:15:34.920 So each system has its own limitations. Getting back to 0:15:34.960 --> 0:15:37.360 the optical one, um, one of the other things you 0:15:37.400 --> 0:15:39.560 have to remember is that in order to really capture 0:15:40.480 --> 0:15:45.560 a a physical object moving through three D space and 0:15:45.680 --> 0:15:51.280 to replicate that in virtual space, you need multiple cameras 0:15:51.440 --> 0:15:55.080 in that system. Because a single camera, assuming that's a 0:15:55.160 --> 0:15:59.480 regular video or film camera, something that does not have 0:15:59.680 --> 0:16:04.680 three capability, pointing that at an object. It's creating a 0:16:04.800 --> 0:16:08.280 two dimensional image of something that's moving in three dimensions. 0:16:09.120 --> 0:16:14.400 The camera can't necessarily tell where movements are happening within 0:16:14.920 --> 0:16:19.720 the depth frame of of that of that image. Right, So, 0:16:19.760 --> 0:16:22.360 if someone's moving in such a way where let's say 0:16:22.360 --> 0:16:25.040 they're moving their head where it would be bobbing closer 0:16:25.080 --> 0:16:31.120 to the camera, Uh, unless the size of the the 0:16:31.200 --> 0:16:34.680 sensors is such that something that subtle could be picked 0:16:34.720 --> 0:16:37.640 up by the camera system, you would lose that information. 0:16:38.760 --> 0:16:41.480 So what you need are multiple cameras on the same 0:16:41.600 --> 0:16:45.000 object so that you can compare that data from the 0:16:45.040 --> 0:16:48.560 multiple angles to tell how this object is really moving 0:16:48.600 --> 0:16:51.560 through this three dimensional space. So it's kind of like 0:16:51.840 --> 0:16:55.400 the idea of having parallax with two eyes. You know, 0:16:55.440 --> 0:16:59.040 our eyes are offset, so by looking at an object, 0:16:59.400 --> 0:17:02.120 we can tell how far away it is in part 0:17:02.680 --> 0:17:07.160 because of parallax. Uh. We also have other visual cues 0:17:07.160 --> 0:17:09.119 that tell us about how far something is, you know, 0:17:09.320 --> 0:17:11.880 things like how tall it is in relation to where 0:17:11.880 --> 0:17:13.480 we are that kind of thing, or how tall it 0:17:13.520 --> 0:17:15.400 is in relation to other objects that are within our 0:17:15.480 --> 0:17:19.080 frame of vision. But parallax is very important. Same sort 0:17:19.080 --> 0:17:21.400 of thing. With these optical systems, you would have multiple 0:17:21.440 --> 0:17:25.920 cameras set up to try and capture the information that's 0:17:25.960 --> 0:17:29.280 going on in the frame so that you could tell 0:17:29.320 --> 0:17:33.160 exactly how it's moving through that three dimensional space. Yeah, 0:17:33.240 --> 0:17:37.000 it seems like um. In order to capture the correct perspective, 0:17:37.359 --> 0:17:40.159 you need that additional information, even though you may not 0:17:40.240 --> 0:17:43.879 necessarily see it. UM. It helps the the animator do that, 0:17:44.160 --> 0:17:47.360 and the optical system to allows you to work with 0:17:47.760 --> 0:17:51.440 more than one actor um, which was not really an 0:17:51.440 --> 0:17:56.440 option with some of the earlier systems. So in other words, 0:17:56.440 --> 0:17:59.879 you can, although it requires more equipment, you know, just 0:18:00.000 --> 0:18:03.840 simply out of necessity, the optical system is really affording 0:18:04.080 --> 0:18:08.280 the animators a an opportunity to use a greater amount 0:18:08.320 --> 0:18:12.520 of information um both you know, from the different the 0:18:12.600 --> 0:18:16.280 different points of data they're getting from a single actor, 0:18:16.680 --> 0:18:20.479 but from multiple actors on the set simultaneously, which enables 0:18:20.520 --> 0:18:24.240 them to to create more complex work right and UH. 0:18:24.400 --> 0:18:27.919 This also gives us a good example of how the 0:18:27.960 --> 0:18:32.720 optical motion capture systems are a passive system because you 0:18:32.760 --> 0:18:35.159 have these sensors you're wearing that are not necessarily or 0:18:35.160 --> 0:18:39.000 not even sensors. They're they're reflective markers that you're wearing. 0:18:39.040 --> 0:18:42.880 They aren't connected to any sort of electronic components at all, 0:18:43.320 --> 0:18:47.960 versus the active systems like the electromagnetic one, where you 0:18:48.040 --> 0:18:52.560 are generating data by moving through a magnetic field and 0:18:52.600 --> 0:18:55.600 you have these big cables attached to it. Uh. With 0:18:55.640 --> 0:18:58.520 the optical motion capture systems. Another thing that's kind of interesting, 0:18:58.600 --> 0:19:00.880 I think is that a lot of least the early ones, 0:19:01.320 --> 0:19:06.520 the cameras would have infrared l ed s uh so admitters, 0:19:06.560 --> 0:19:10.959 really that we're emitting infrared lights. That's outside our our 0:19:11.040 --> 0:19:15.160 visible spectrum. We cannot see infrared light. But by putting 0:19:15.200 --> 0:19:18.320 an infrared filter on the camera, you could have the 0:19:18.359 --> 0:19:21.879 camera pick up reflections of infrared light. And that was 0:19:21.920 --> 0:19:25.879 a way of helping to identify the sensors that you 0:19:25.960 --> 0:19:29.240 had put on the actor. The actors, the sensors would 0:19:29.280 --> 0:19:33.240 be reflective specifically so that the infrared light would reflect 0:19:33.280 --> 0:19:37.080 back toward the camera and give the most accurate rendering 0:19:37.160 --> 0:19:41.000 of what's going on at any given moment within a scene. 0:19:41.560 --> 0:19:45.520 So um, yeah, it's another way of making sure that 0:19:45.680 --> 0:19:48.560 the data being captured is as precise as possible. I 0:19:48.560 --> 0:19:50.200 mean that is, of course, the goal is to try 0:19:50.200 --> 0:19:55.240 and recreate the physical movements as truthfully as you possibly 0:19:55.320 --> 0:19:59.880 can given all the limitations involved. Yeah, and if you're 0:20:00.000 --> 0:20:04.600 looking for a real life easy to find an example 0:20:04.680 --> 0:20:08.679 of this, you would look no farther than your local 0:20:08.760 --> 0:20:13.879 video game store. Um, because the Xbox Connect h uses 0:20:14.000 --> 0:20:18.119 very much that that exact uh form of technology. It 0:20:18.320 --> 0:20:21.480 is using an infrared emitter, um, and it has cameras 0:20:21.520 --> 0:20:24.240 that it uses to pick it up uh, the information 0:20:24.440 --> 0:20:27.040 pick the information up that is coming back from what 0:20:27.200 --> 0:20:29.720 is being reflected around the room, and anybody who who 0:20:29.760 --> 0:20:33.080 has one is also aware that lighting is very much 0:20:33.119 --> 0:20:36.040 an issue. Um. The way that were room is let 0:20:36.119 --> 0:20:39.160 affects the information that the Connect is able to refer 0:20:39.200 --> 0:20:42.200 to the Xbox. Now, it's not, while it is sophisticated, 0:20:42.280 --> 0:20:44.399 is not as sophisticated as the kind of equipment that 0:20:44.440 --> 0:20:47.159 they might use in making a movie or making a 0:20:47.280 --> 0:20:51.000 video game. But it is very very similar technology, and 0:20:51.000 --> 0:20:53.960 in some ways I would argue that it's more sophisticated 0:20:53.960 --> 0:20:57.080 than some of those early UH systems simply because it 0:20:57.160 --> 0:21:00.800 is able to capture a lot of information uh, Whereas 0:21:00.880 --> 0:21:03.560 you know, the very early optical systems were only using 0:21:03.760 --> 0:21:08.480 a handful of data points. So um, it's it's a 0:21:08.520 --> 0:21:11.800 pretty neat device. Um, you know, not only used for gaming. 0:21:11.800 --> 0:21:13.679 Now the hacker community has fallen in love with it 0:21:13.720 --> 0:21:16.280 too because it can do so much and can be 0:21:16.359 --> 0:21:20.000 used for so many things and is you know, fairly inexpensive. Yeah. 0:21:20.000 --> 0:21:22.680 The cool thing about the Connect is that rather than 0:21:22.720 --> 0:21:25.480 have to obviously, if you've if you've ever played an 0:21:25.640 --> 0:21:27.959 Xbox with the connect, you know, you don't have to 0:21:28.040 --> 0:21:31.000 go out and buy a snug body suit covered in 0:21:31.040 --> 0:21:33.800 reflective markers in order to play. I mean, it doesn't hurt, 0:21:34.600 --> 0:21:36.720 but uh, you know, if you're if you can pull 0:21:36.760 --> 0:21:39.000 that look off. There a very few of us who can. 0:21:39.280 --> 0:21:42.560 I count myself among them. But you don't have to 0:21:42.600 --> 0:21:45.080 do that because what it's doing is it's actually projecting 0:21:45.200 --> 0:21:50.120 essentially a grid, uh in infrared light, so you can't 0:21:50.160 --> 0:21:52.880 see the grid, but it's being projected into the room. 0:21:53.040 --> 0:21:56.760 And then when you move uh within the space, you 0:21:56.800 --> 0:21:59.440 are deforming that grid. You know, the camera that's picking 0:21:59.520 --> 0:22:04.560 up the the reflections of that infrared light can detect 0:22:04.600 --> 0:22:08.359 when the grid's being deformed by a physical object interrupting 0:22:08.400 --> 0:22:11.440 the grid. So as you move, you interrupt different parts 0:22:11.480 --> 0:22:13.800 of the grid, and it can start to interpret those 0:22:13.880 --> 0:22:19.600 as motions and commands. It's not, uh, it's not as 0:22:19.640 --> 0:22:22.800 precise as what we're talking about with the optical systems 0:22:22.840 --> 0:22:25.560 that are used in movies and video games, uh, to 0:22:25.560 --> 0:22:29.200 to create them, that is, not to to play them. Um, 0:22:29.240 --> 0:22:32.200 it's not as precise as those, but it also has 0:22:32.280 --> 0:22:35.520 other elements that help balance it out, Like it has 0:22:36.200 --> 0:22:42.120 regular optical cameras that can have some other software that 0:22:42.520 --> 0:22:46.640 aids it in recognizing things like facial recognition software, which 0:22:46.640 --> 0:22:50.960 does not necessarily rely upon that infrared grid. It relies 0:22:51.000 --> 0:22:55.680 more on the traditional camera functions, but has the software 0:22:55.720 --> 0:23:00.280 included that, let's the the programs within recognize who is 0:23:00.320 --> 0:23:04.280 standing in front of it, so that combination, uh increases 0:23:04.320 --> 0:23:06.560 the precision, which of course is very important whenever you're 0:23:06.560 --> 0:23:09.120 playing a game. I mean, anyone who's played any sort 0:23:09.200 --> 0:23:12.560 of game where you're using a faulty controller, or it's 0:23:12.600 --> 0:23:16.479 just a system that hasn't been fully uh it's not 0:23:16.520 --> 0:23:19.080 finished yet, it's just in prototype stage or whatever, you 0:23:19.119 --> 0:23:22.359 may have noticed that it could be very frustrating to 0:23:22.480 --> 0:23:26.399 try and control something where the actual controller is not 0:23:27.440 --> 0:23:31.239 as responsive as you would hope. It's um not a 0:23:31.280 --> 0:23:34.880 fun experience. But anyway, that is kind of related to 0:23:34.960 --> 0:23:41.560 this whole motion capture technology. UM, I'm sorry what you 0:23:41.320 --> 0:23:43.639 were You look like you have something to say, Well, no, 0:23:43.760 --> 0:23:46.320 I was, I was going to say that. Um, you know, 0:23:46.359 --> 0:23:51.240 we really hadn't other than my earlier statement about sports. UM, 0:23:51.240 --> 0:23:52.879 you know, we've we've been talking about it in an 0:23:53.040 --> 0:23:59.120 entertain amount entertainment about the the ability to capture motion 0:23:59.200 --> 0:24:02.800 to make care act is more realistic. And um, that 0:24:02.800 --> 0:24:05.479 that is exactly what they want to do when they 0:24:05.480 --> 0:24:08.399 are using this in sports medicine. UM. Jonathan alluded to 0:24:08.480 --> 0:24:13.119 earlier the difficulty in UH and capturing all the little 0:24:13.160 --> 0:24:18.200 subtle motions that go into UM into a Major League 0:24:18.200 --> 0:24:22.080 baseball players pitching. UM. And you know when somebody, when 0:24:22.080 --> 0:24:27.359 somebody gets hurt, UM, sometimes they go through uh extensive surgery. 0:24:27.920 --> 0:24:31.399 The Tommy John procedures is UH famous. You know, they 0:24:31.600 --> 0:24:35.720 do a ligament transplant to to help rebuild a picture's elbow, 0:24:36.119 --> 0:24:39.119