1 00:00:00,520 --> 00:00:03,160 Speaker 1: Brought to you by the two thousand and twelve Toyota Camera. 2 00:00:05,880 --> 00:00:08,920 Speaker 1: Get in touch with technology with tech Stuff from how 3 00:00:08,960 --> 00:00:17,000 Speaker 1: stuff works dot com. Hello again, everyone, and welcome to 4 00:00:17,040 --> 00:00:19,000 Speaker 1: tech stuff. My name is Chris Poulette, and I'm an 5 00:00:19,079 --> 00:00:21,720 Speaker 1: editor at how stuff works dot com. Sitting across from 6 00:00:21,720 --> 00:00:24,440 Speaker 1: me as always as senior writer Jonathan Strickland. Hey there, 7 00:00:25,200 --> 00:00:28,640 Speaker 1: So today we thought we'd talk a bit about a 8 00:00:28,760 --> 00:00:33,720 Speaker 1: type of performance that is relatively new as far as 9 00:00:33,760 --> 00:00:37,680 Speaker 1: performance goes. Uh, something that uh, I guess this falls 10 00:00:37,680 --> 00:00:41,080 Speaker 1: into our movie making category, but it's also something that's 11 00:00:41,120 --> 00:00:44,120 Speaker 1: been used in things like video games and and other 12 00:00:44,159 --> 00:00:48,640 Speaker 1: forms of media as well. Motion capture. Yeah, you'll even 13 00:00:48,680 --> 00:00:52,720 Speaker 1: see it in uh, in sports, they've been talking about 14 00:00:52,720 --> 00:00:55,120 Speaker 1: this for a while now. And if you've ever seen 15 00:00:55,480 --> 00:01:01,040 Speaker 1: the making of a a video or a game, um, 16 00:01:01,160 --> 00:01:05,400 Speaker 1: or you know, even in sports rehabilitation in medicine, um, 17 00:01:05,440 --> 00:01:08,480 Speaker 1: they where the people are wearing dots, little white dots 18 00:01:08,520 --> 00:01:12,880 Speaker 1: all over their clothing and sometimes their faces and hands. UM, 19 00:01:12,920 --> 00:01:15,679 Speaker 1: that's probably what they were doing. Either that or they 20 00:01:15,760 --> 00:01:18,960 Speaker 1: just really like stickers. Yeah, yeah, I mean, who doesn't. 21 00:01:19,120 --> 00:01:22,880 Speaker 1: I remember being very competitive in elementary school in order 22 00:01:22,920 --> 00:01:25,800 Speaker 1: to get a sticker. And also this is a tangent 23 00:01:25,880 --> 00:01:29,120 Speaker 1: but a true story. I got a gold star sticker 24 00:01:29,520 --> 00:01:32,920 Speaker 1: uh just last month awes from Tracy, the head of 25 00:01:32,920 --> 00:01:37,959 Speaker 1: our our site. So anyway, um, yeah, motion capture. Actually, 26 00:01:38,000 --> 00:01:40,759 Speaker 1: there are a lot of different terms that you can 27 00:01:40,920 --> 00:01:44,800 Speaker 1: use in this in this realm Uh, motion capture or 28 00:01:44,840 --> 00:01:47,600 Speaker 1: mo cap is probably the one I hear the most frequently, 29 00:01:47,640 --> 00:01:53,120 Speaker 1: but also things like performance animation, performance capture, digital puppetry, 30 00:01:53,280 --> 00:01:57,080 Speaker 1: real time animation, motion scanning, which is really more of 31 00:01:57,080 --> 00:02:01,000 Speaker 1: a proprietary thing, but these are The concept is pretty 32 00:02:01,040 --> 00:02:03,000 Speaker 1: much the same across the board. The idea is to 33 00:02:03,240 --> 00:02:08,919 Speaker 1: capture the physical representation of something and then converted into 34 00:02:09,000 --> 00:02:12,799 Speaker 1: a virtual format. So usually it's something that's in motion, 35 00:02:12,840 --> 00:02:15,560 Speaker 1: but it's not always that way. Uh, since you know 36 00:02:15,600 --> 00:02:19,280 Speaker 1: we're talking about motion capture, that makes sense. But you're 37 00:02:19,320 --> 00:02:23,359 Speaker 1: trying to get uh, translate something that is moving through 38 00:02:23,440 --> 00:02:28,560 Speaker 1: real space into a digital format. And uh, there's different 39 00:02:28,600 --> 00:02:30,000 Speaker 1: ways to do this. I mean, you could do it 40 00:02:30,080 --> 00:02:32,639 Speaker 1: the really hard way, which is where you study something 41 00:02:32,680 --> 00:02:35,680 Speaker 1: and then you try to recreate it, uh, either by 42 00:02:35,760 --> 00:02:38,560 Speaker 1: hand or by or digitally, you know, by by programming 43 00:02:39,080 --> 00:02:43,400 Speaker 1: movements into an animated figure. But this is an idea 44 00:02:43,400 --> 00:02:46,000 Speaker 1: that kind of takes that step out where you are 45 00:02:46,560 --> 00:02:50,800 Speaker 1: directly porting the movements. Uh, something is making within physical 46 00:02:50,800 --> 00:02:55,200 Speaker 1: space into virtual space. Yeah, there was an early technique, um. 47 00:02:55,240 --> 00:02:56,960 Speaker 1: And of course this is this is all an attempt 48 00:02:57,040 --> 00:03:01,720 Speaker 1: to get as real as you can with of animation. UM. 49 00:03:02,400 --> 00:03:06,240 Speaker 1: And one of the earlier techniques that that was sort 50 00:03:06,280 --> 00:03:10,359 Speaker 1: of a predecessor to this is called rotoscoping. Uh. Ralph 51 00:03:10,400 --> 00:03:13,400 Speaker 1: Box She's Lord of the Rings had a lot of 52 00:03:13,520 --> 00:03:17,520 Speaker 1: rotoscoping in it. Well, what happens is, um, in that 53 00:03:17,600 --> 00:03:21,440 Speaker 1: case is that a real uh, real human being goes 54 00:03:21,520 --> 00:03:24,480 Speaker 1: through the motions and they act through the parts that 55 00:03:24,520 --> 00:03:26,760 Speaker 1: are that you're going to see in the animation. They 56 00:03:26,760 --> 00:03:30,600 Speaker 1: shoot that on film, yes, yes, and then the the 57 00:03:30,639 --> 00:03:34,640 Speaker 1: animators basically are looking at that and are drawing more 58 00:03:34,720 --> 00:03:37,160 Speaker 1: or less on top of that. They see a projection 59 00:03:37,160 --> 00:03:39,800 Speaker 1: of that, and they are drawing, uh, the animation over 60 00:03:39,840 --> 00:03:44,080 Speaker 1: that to capture the way that person's body looks. And 61 00:03:44,880 --> 00:03:47,760 Speaker 1: this this was famous. You know, the Disney studios were 62 00:03:47,760 --> 00:03:50,120 Speaker 1: famous for this. We're studying models and then they would 63 00:03:50,120 --> 00:03:54,600 Speaker 1: do the rotoscoping technique to to try to make their uh, 64 00:03:55,000 --> 00:03:58,480 Speaker 1: their characters look more realistic. Yeah. And there are some artists, 65 00:03:58,560 --> 00:04:02,480 Speaker 1: like I said, like box She who famously would leave 66 00:04:02,560 --> 00:04:06,120 Speaker 1: the film image as part of the animation, so that 67 00:04:06,160 --> 00:04:09,120 Speaker 1: you had this this weird effect where the thing you 68 00:04:09,160 --> 00:04:13,360 Speaker 1: were looking at was part uh well quote unquote real 69 00:04:13,400 --> 00:04:17,240 Speaker 1: image and part animated image, which was it was an 70 00:04:17,320 --> 00:04:21,200 Speaker 1: artistic choice, uh, definitely something that was not meant to 71 00:04:21,200 --> 00:04:24,280 Speaker 1: to necessarily fool you into thinking, oh, well, that animated 72 00:04:24,360 --> 00:04:26,960 Speaker 1: character is moving very realistically. It was done on purpose, 73 00:04:27,560 --> 00:04:29,560 Speaker 1: but it was. That's what I always think of when 74 00:04:29,600 --> 00:04:32,119 Speaker 1: I think rhodoscoping, as I just think of the different 75 00:04:32,200 --> 00:04:34,040 Speaker 1: box Sheet films, but in particular I think of his 76 00:04:34,160 --> 00:04:37,640 Speaker 1: Lord of the Rings adaptation um, which, as I recall, 77 00:04:37,800 --> 00:04:42,800 Speaker 1: ended halfway through the Two Towers. So anyway, that's just 78 00:04:42,839 --> 00:04:45,240 Speaker 1: bringing back memories. But yeah, that was that was sort 79 00:04:45,240 --> 00:04:50,560 Speaker 1: of a precursor to motion capture. Motion capture itself. There 80 00:04:50,560 --> 00:04:55,599 Speaker 1: are many different ways of achieving this. For example, there 81 00:04:55,680 --> 00:05:00,159 Speaker 1: were it's not used very frequently now, but there were 82 00:05:00,800 --> 00:05:04,960 Speaker 1: mechanical systems where you had sensors that would be attached 83 00:05:05,000 --> 00:05:10,640 Speaker 1: to specific joints uh that would relay movement. And usually 84 00:05:10,640 --> 00:05:12,839 Speaker 1: it was kind of like a like an actor would 85 00:05:12,839 --> 00:05:19,039 Speaker 1: wear a physical metallic skeleton type device that would have 86 00:05:19,160 --> 00:05:22,800 Speaker 1: the sensors attached to the various joints and as the 87 00:05:22,839 --> 00:05:27,719 Speaker 1: actor moved, the sensors would register the changes in motion 88 00:05:28,600 --> 00:05:33,160 Speaker 1: in this metallic skeleton and UH, and that would be 89 00:05:33,200 --> 00:05:38,560 Speaker 1: relayed through usually cables to a computer system that would 90 00:05:39,120 --> 00:05:42,080 Speaker 1: measure these or take the measurements from the sensors and 91 00:05:42,080 --> 00:05:45,960 Speaker 1: translated into movements for the virtual character. UH. It's very 92 00:05:46,040 --> 00:05:49,839 Speaker 1: limiting this particular system. There was another one that was 93 00:05:49,920 --> 00:05:55,280 Speaker 1: a little more versatile, which was used electro magnets. And 94 00:05:55,320 --> 00:05:58,640 Speaker 1: in this case you talked about sensors that would be 95 00:05:58,680 --> 00:06:02,279 Speaker 1: attached by really thin cables that again would go to 96 00:06:02,520 --> 00:06:06,080 Speaker 1: a computer, and there'd be a magnetic field and by 97 00:06:06,160 --> 00:06:10,440 Speaker 1: moving through this magnetic field, the sensors would pick up alterations. 98 00:06:10,440 --> 00:06:13,000 Speaker 1: You know, it would you know, moving through magnetic field, 99 00:06:13,000 --> 00:06:16,880 Speaker 1: you would get little electrical changes. We we've talked a 100 00:06:16,880 --> 00:06:22,880 Speaker 1: lot about electricity magnetism in general, moving through UH, Fluctuating 101 00:06:22,880 --> 00:06:27,039 Speaker 1: a magnetic field can induce electricity through a conductor, or 102 00:06:27,400 --> 00:06:30,800 Speaker 1: putting electricity through a conductor can induce a magnetic field. 103 00:06:31,000 --> 00:06:34,240 Speaker 1: So anyway, by moving these sensors through the magnetic field, 104 00:06:34,440 --> 00:06:38,359 Speaker 1: it would create these electronic fluctuations that would then be 105 00:06:38,480 --> 00:06:43,359 Speaker 1: measured and translated into movement, and again this was a 106 00:06:43,560 --> 00:06:47,880 Speaker 1: fairly effective way of picking up movements. It actually didn't 107 00:06:48,000 --> 00:06:50,960 Speaker 1: use as many points of contact as the optical systems 108 00:06:50,960 --> 00:06:52,919 Speaker 1: that we mostly think about. That was the kind that 109 00:06:53,000 --> 00:06:55,599 Speaker 1: Chris was referring to early with all the dots on 110 00:06:55,640 --> 00:06:58,960 Speaker 1: the person. Those systems tend to have lots and lots 111 00:06:58,960 --> 00:07:02,800 Speaker 1: and lots of points of ref. The electro magnet ones 112 00:07:03,279 --> 00:07:06,799 Speaker 1: didn't tend to have as many points of reference because 113 00:07:07,320 --> 00:07:10,640 Speaker 1: the the software side of it, because you know, we 114 00:07:10,680 --> 00:07:12,520 Speaker 1: do have a hardware and a software side to this. 115 00:07:12,880 --> 00:07:16,360 Speaker 1: The software side would assume that the joints that these 116 00:07:16,400 --> 00:07:20,440 Speaker 1: sensors were attached to behaved the way they normally would 117 00:07:20,480 --> 00:07:24,239 Speaker 1: in humans, and that they don't have complete freedom of movement. 118 00:07:24,440 --> 00:07:28,600 Speaker 1: Most of us are not multi jointed in every joint, 119 00:07:28,720 --> 00:07:30,800 Speaker 1: so we can't you know, that we have a limitation 120 00:07:30,840 --> 00:07:33,080 Speaker 1: on how far we can move in certain directions with 121 00:07:33,120 --> 00:07:36,440 Speaker 1: these various joints. So taking that into account, you didn't 122 00:07:36,480 --> 00:07:40,040 Speaker 1: have to have sensors all over the body. You would 123 00:07:40,240 --> 00:07:41,920 Speaker 1: just have them in a few places, which was good 124 00:07:41,960 --> 00:07:44,520 Speaker 1: considering that there were these thick cables attached to the 125 00:07:44,600 --> 00:07:49,080 Speaker 1: sensors and then once you were done moving, then the 126 00:07:49,480 --> 00:07:52,720 Speaker 1: all that data would get be captured within the system 127 00:07:52,800 --> 00:07:56,600 Speaker 1: and could then be rendered into animation. Although this was 128 00:07:56,640 --> 00:07:59,200 Speaker 1: also a way that you could do real time animation 129 00:07:59,320 --> 00:08:02,720 Speaker 1: or digital a tree. Uh, it's not that different from 130 00:08:02,720 --> 00:08:06,160 Speaker 1: controlling a video game character with a controller. It's sort 131 00:08:06,160 --> 00:08:08,320 Speaker 1: of the same principle, except in this case the the 132 00:08:08,560 --> 00:08:10,800 Speaker 1: video game controller. Instead of it being something you hold 133 00:08:10,800 --> 00:08:15,120 Speaker 1: in your hands, it's something you were actually wearing. And uh, 134 00:08:15,240 --> 00:08:17,600 Speaker 1: I've seen plenty of instances of this. If you've ever 135 00:08:17,600 --> 00:08:21,400 Speaker 1: seen Turtle Talk with Crush over at Disney, that's what 136 00:08:21,480 --> 00:08:23,280 Speaker 1: they use. They use a digital you know, they use 137 00:08:23,320 --> 00:08:27,440 Speaker 1: digital poetry, and it's awesome by the way, I love that. 138 00:08:28,360 --> 00:08:32,200 Speaker 1: Well it uh, it would also seem that, um, you 139 00:08:32,200 --> 00:08:36,559 Speaker 1: would need to be aware of where those cables were going, 140 00:08:37,080 --> 00:08:39,280 Speaker 1: and it would it would also affect the way that 141 00:08:39,280 --> 00:08:41,600 Speaker 1: you would move. You wouldn't move as naturally if you 142 00:08:41,640 --> 00:08:44,360 Speaker 1: were wearing something like that as if you were, you know, 143 00:08:44,760 --> 00:08:48,920 Speaker 1: unencumbered by by that, which, um, sort of I think 144 00:08:49,000 --> 00:08:52,400 Speaker 1: would lend itself to to an upgrade, which is I 145 00:08:52,440 --> 00:08:57,839 Speaker 1: think why they were so keen on Well, it's also 146 00:08:58,120 --> 00:09:00,920 Speaker 1: also that's very true. It did limit what you could do, 147 00:09:01,000 --> 00:09:02,720 Speaker 1: it could live, It would limit your movement. I mean, 148 00:09:02,800 --> 00:09:05,000 Speaker 1: we've got these big cables attached to you. You You obviously 149 00:09:05,040 --> 00:09:08,559 Speaker 1: you can't just move freely within a space. Um, So 150 00:09:08,640 --> 00:09:11,679 Speaker 1: it did put some limitations on you. Their limitations to 151 00:09:11,720 --> 00:09:15,560 Speaker 1: the optical systems too, but we'll get into that. The 152 00:09:14,800 --> 00:09:19,320 Speaker 1: uh the other problem was that the sampling rate for 153 00:09:19,400 --> 00:09:22,360 Speaker 1: the magnetic systems was not as high as it is 154 00:09:22,400 --> 00:09:24,800 Speaker 1: for optical systems. And by sampling rate, what I mean 155 00:09:24,880 --> 00:09:28,240 Speaker 1: is that this the entire system as a whole, is 156 00:09:28,280 --> 00:09:33,520 Speaker 1: taking little measurements of from the sensors of you know, 157 00:09:33,559 --> 00:09:37,200 Speaker 1: the orientation of those sensors within the space, and it 158 00:09:37,280 --> 00:09:42,000 Speaker 1: does that several times every second. But the sample rate 159 00:09:42,080 --> 00:09:46,160 Speaker 1: of the magnetic motion capture systems was much lower than 160 00:09:46,240 --> 00:09:49,400 Speaker 1: what it was for than what it would be if 161 00:09:49,400 --> 00:09:51,480 Speaker 1: you were to use an optical system. So you're not 162 00:09:51,559 --> 00:09:55,520 Speaker 1: getting data as frequently. I mean still several times a second, 163 00:09:55,559 --> 00:09:59,000 Speaker 1: but it's not as precise as the optical system. So 164 00:09:59,520 --> 00:10:01,720 Speaker 1: not only where you limited in the kind of movements 165 00:10:01,760 --> 00:10:05,640 Speaker 1: you can make because you had these major cables attached 166 00:10:05,679 --> 00:10:11,520 Speaker 1: to you, but also you couldn't get really minute precise 167 00:10:12,480 --> 00:10:16,040 Speaker 1: measurements on every kind of movement, So it wasn't good 168 00:10:16,040 --> 00:10:19,480 Speaker 1: for things like sports. So, you know, something like throwing 169 00:10:19,480 --> 00:10:22,200 Speaker 1: a pitch in baseball, there are a lot of movements, 170 00:10:22,200 --> 00:10:25,160 Speaker 1: a little tiny motions that are involved in that. I mean, 171 00:10:25,360 --> 00:10:30,880 Speaker 1: anyone who's watched slow motion footage of a professional baseball 172 00:10:30,920 --> 00:10:33,800 Speaker 1: pitcher throwing a pitch, you can see that there are 173 00:10:33,840 --> 00:10:38,880 Speaker 1: some incredibly subtle movements that are involved in that. And uh, 174 00:10:38,920 --> 00:10:41,960 Speaker 1: and it takes place over a very short period of time. 175 00:10:42,000 --> 00:10:45,600 Speaker 1: I mean, it's a very fast thing to to to measure. 176 00:10:46,600 --> 00:10:50,400 Speaker 1: Using the magnetic motion capture system, you would probably one 177 00:10:51,160 --> 00:10:53,640 Speaker 1: slow the person down because they have all these cables 178 00:10:53,679 --> 00:10:56,440 Speaker 1: attached to them, and to not get enough data to 179 00:10:56,920 --> 00:11:00,520 Speaker 1: give an accurate representation of what had happened in the 180 00:11:00,600 --> 00:11:04,480 Speaker 1: virtual format. So if you were to say, create a 181 00:11:04,559 --> 00:11:07,520 Speaker 1: video game, a baseball video game, the picture would not 182 00:11:07,679 --> 00:11:13,760 Speaker 1: necessarily behave properly if all you did was directly port 183 00:11:13,800 --> 00:11:17,560 Speaker 1: the data you got from the motion capture into the game. Yeah. 184 00:11:17,559 --> 00:11:20,640 Speaker 1: Another drawback of the mechanical systems like that too, is 185 00:11:20,679 --> 00:11:23,760 Speaker 1: that that it's um it's the kind of system that 186 00:11:23,800 --> 00:11:27,040 Speaker 1: not only is cumbersome and inaccurate, but it has to 187 00:11:27,040 --> 00:11:31,200 Speaker 1: be calibrated fairly frequently. UM. And you know, there there's 188 00:11:31,200 --> 00:11:33,640 Speaker 1: a there's some work that you can do with this. 189 00:11:33,760 --> 00:11:37,920 Speaker 1: With the optical systems that they began to introduce UM, 190 00:11:37,960 --> 00:11:43,040 Speaker 1: you know, generally became an upgrade UM. The only there 191 00:11:43,120 --> 00:11:47,760 Speaker 1: is one big advantage that the mechanical systems do have, though, 192 00:11:47,800 --> 00:11:51,400 Speaker 1: and that is that light. The lighting will not necessarily 193 00:11:51,400 --> 00:11:54,880 Speaker 1: interfere with the different points of motion that are captured 194 00:11:54,880 --> 00:11:57,880 Speaker 1: by the mechanical system UM. And that can be an 195 00:11:57,880 --> 00:12:01,960 Speaker 1: issue with the optical systems UM. You know, because that's 196 00:12:02,080 --> 00:12:04,640 Speaker 1: that's why UM, they will be wearing the people, the 197 00:12:04,679 --> 00:12:08,560 Speaker 1: actors who will be UM having their motions captured by 198 00:12:08,600 --> 00:12:11,760 Speaker 1: the system, will be wearing you know, those bright dots 199 00:12:11,920 --> 00:12:14,400 Speaker 1: so that the computer can pick up on that. And 200 00:12:14,440 --> 00:12:16,440 Speaker 1: at the beginning, and these are these early systems, there 201 00:12:16,440 --> 00:12:20,720 Speaker 1: were only so many points action points that they could capture. UM. 202 00:12:20,800 --> 00:12:23,400 Speaker 1: They were very limited in what they could do at first, 203 00:12:23,520 --> 00:12:25,880 Speaker 1: but still you know, somewhat of an upgrade over the 204 00:12:25,920 --> 00:12:30,400 Speaker 1: mechanical Yeah. It also limited what you could have in 205 00:12:30,440 --> 00:12:34,040 Speaker 1: the background, obviously, because you could not have anything that 206 00:12:34,200 --> 00:12:38,079 Speaker 1: was going to be of a similar shade. Uh. You know, 207 00:12:38,160 --> 00:12:42,199 Speaker 1: usually where you're talking about reflective white substance used as 208 00:12:42,240 --> 00:12:47,440 Speaker 1: the um the points of of uh articulations. So the 209 00:12:47,600 --> 00:12:50,600 Speaker 1: little like white stickers is like what you were saying, 210 00:12:50,640 --> 00:12:53,199 Speaker 1: Chris Um, you couldn't have anything like that in the 211 00:12:53,240 --> 00:12:57,840 Speaker 1: background because it would confuse the optical system. So that's 212 00:12:57,880 --> 00:13:01,439 Speaker 1: why a lot of these motion capture scenes are shot 213 00:13:01,440 --> 00:13:04,760 Speaker 1: against a blue screen or green screen. It's so that 214 00:13:05,040 --> 00:13:08,720 Speaker 1: the background does not in any way interfere with the 215 00:13:08,800 --> 00:13:11,240 Speaker 1: motion capture. So if you've ever seen behind the scenes 216 00:13:11,240 --> 00:13:13,280 Speaker 1: footage of The Lord of the Rings movies is a 217 00:13:13,280 --> 00:13:18,120 Speaker 1: great example with Andy Serkis as Gollum or Sniegel if 218 00:13:18,160 --> 00:13:22,480 Speaker 1: you prefer, but he's wearing you know, a tight like 219 00:13:22,760 --> 00:13:27,360 Speaker 1: skin tight suit with these little white uh circles all 220 00:13:27,400 --> 00:13:30,800 Speaker 1: over it. Those are the points that the cameras track 221 00:13:31,280 --> 00:13:36,080 Speaker 1: to create the the performance of Gollum slash Sniegel. So 222 00:13:36,480 --> 00:13:39,880 Speaker 1: the performance is something that's being created not only by 223 00:13:40,120 --> 00:13:43,840 Speaker 1: the actor but also the animators because not We should 224 00:13:43,880 --> 00:13:47,240 Speaker 1: also point out that the motion capture stuff rarely is 225 00:13:47,320 --> 00:13:54,120 Speaker 1: motion capture uh completely. Uh. There's there's rarely a moment 226 00:13:54,120 --> 00:13:56,520 Speaker 1: where you don't have an animator step in and tweak 227 00:13:56,559 --> 00:14:01,520 Speaker 1: it somehow, like Uh, you don't normally have someone create 228 00:14:01,559 --> 00:14:06,840 Speaker 1: a physical performance and that physical performance is completely without 229 00:14:06,920 --> 00:14:10,719 Speaker 1: any tinkering represented in the final product. I mean it 230 00:14:11,040 --> 00:14:14,760 Speaker 1: can happen, there are instances of it, but it's more frequently, uh, 231 00:14:14,800 --> 00:14:18,439 Speaker 1: something where the motion capture performance goes to the animator, 232 00:14:18,480 --> 00:14:21,720 Speaker 1: who can then tweak things if the performance is not 233 00:14:21,840 --> 00:14:24,840 Speaker 1: exactly what needs to be, which is kind of nice. 234 00:14:25,240 --> 00:14:28,480 Speaker 1: You don't necessarily have that luxury with flesh and blood actors. 235 00:14:29,280 --> 00:14:33,000 Speaker 1: That's that's true. That's true. Well, especially with the earlier systems, 236 00:14:33,080 --> 00:14:38,440 Speaker 1: especially the electromagnetic systems. Uh, those were really noisy, not 237 00:14:38,520 --> 00:14:42,120 Speaker 1: literally noisy, but but digital noise. They weren't They weren't 238 00:14:42,160 --> 00:14:46,240 Speaker 1: really highly accurate. Um. The optical systems are are far 239 00:14:46,440 --> 00:14:49,920 Speaker 1: cleaner and give them more accurate representation. But you know there, 240 00:14:50,000 --> 00:14:52,720 Speaker 1: that's it's sort of falls in the realm of artistic license. 241 00:14:52,720 --> 00:14:55,200 Speaker 1: I would think, um, where they need to go in 242 00:14:55,240 --> 00:14:59,000 Speaker 1: and make subtle adjustments to make it look the way 243 00:14:59,000 --> 00:15:01,560 Speaker 1: they want it to look. Up. I should also point out, 244 00:15:01,720 --> 00:15:04,240 Speaker 1: now you just reminded me of something else another drawback 245 00:15:04,280 --> 00:15:07,720 Speaker 1: to the electromagnetic systems, which was you couldn't have anything 246 00:15:07,760 --> 00:15:11,840 Speaker 1: metal on the set because it would interfere with that 247 00:15:11,880 --> 00:15:15,520 Speaker 1: magnetic field and give incorrect readings to the system. So 248 00:15:15,600 --> 00:15:18,880 Speaker 1: you're you're virtual character would not move in the same 249 00:15:18,920 --> 00:15:20,640 Speaker 1: way as the physical one because there would be some 250 00:15:20,680 --> 00:15:24,440 Speaker 1: interference in that sense. So your set couldn't have anything 251 00:15:24,600 --> 00:15:28,640 Speaker 1: metal in it. The props didn't shouldn't have anything metal 252 00:15:28,800 --> 00:15:31,160 Speaker 1: in them, so that that limited to you as well. 253 00:15:31,240 --> 00:15:34,920 Speaker 1: So each system has its own limitations. Getting back to 254 00:15:34,960 --> 00:15:37,360 Speaker 1: the optical one, um, one of the other things you 255 00:15:37,400 --> 00:15:39,560 Speaker 1: have to remember is that in order to really capture 256 00:15:40,480 --> 00:15:45,560 Speaker 1: a a physical object moving through three D space and 257 00:15:45,680 --> 00:15:51,280 Speaker 1: to replicate that in virtual space, you need multiple cameras 258 00:15:51,440 --> 00:15:55,080 Speaker 1: in that system. Because a single camera, assuming that's a 259 00:15:55,160 --> 00:15:59,480 Speaker 1: regular video or film camera, something that does not have 260 00:15:59,680 --> 00:16:04,680 Speaker 1: three capability, pointing that at an object. It's creating a 261 00:16:04,800 --> 00:16:08,280 Speaker 1: two dimensional image of something that's moving in three dimensions. 262 00:16:09,120 --> 00:16:14,400 Speaker 1: The camera can't necessarily tell where movements are happening within 263 00:16:14,920 --> 00:16:19,720 Speaker 1: the depth frame of of that of that image. Right, So, 264 00:16:19,760 --> 00:16:22,360 Speaker 1: if someone's moving in such a way where let's say 265 00:16:22,360 --> 00:16:25,040 Speaker 1: they're moving their head where it would be bobbing closer 266 00:16:25,080 --> 00:16:31,120 Speaker 1: to the camera, Uh, unless the size of the the 267 00:16:31,200 --> 00:16:34,680 Speaker 1: sensors is such that something that subtle could be picked 268 00:16:34,720 --> 00:16:37,640 Speaker 1: up by the camera system, you would lose that information. 269 00:16:38,760 --> 00:16:41,480 Speaker 1: So what you need are multiple cameras on the same 270 00:16:41,600 --> 00:16:45,000 Speaker 1: object so that you can compare that data from the 271 00:16:45,040 --> 00:16:48,560 Speaker 1: multiple angles to tell how this object is really moving 272 00:16:48,600 --> 00:16:51,560 Speaker 1: through this three dimensional space. So it's kind of like 273 00:16:51,840 --> 00:16:55,400 Speaker 1: the idea of having parallax with two eyes. You know, 274 00:16:55,440 --> 00:16:59,040 Speaker 1: our eyes are offset, so by looking at an object, 275 00:16:59,400 --> 00:17:02,120 Speaker 1: we can tell how far away it is in part 276 00:17:02,680 --> 00:17:07,160 Speaker 1: because of parallax. Uh. We also have other visual cues 277 00:17:07,160 --> 00:17:09,119 Speaker 1: that tell us about how far something is, you know, 278 00:17:09,320 --> 00:17:11,880 Speaker 1: things like how tall it is in relation to where 279 00:17:11,880 --> 00:17:13,480 Speaker 1: we are that kind of thing, or how tall it 280 00:17:13,520 --> 00:17:15,400 Speaker 1: is in relation to other objects that are within our 281 00:17:15,480 --> 00:17:19,080 Speaker 1: frame of vision. But parallax is very important. Same sort 282 00:17:19,080 --> 00:17:21,400 Speaker 1: of thing. With these optical systems, you would have multiple 283 00:17:21,440 --> 00:17:25,920 Speaker 1: cameras set up to try and capture the information that's 284 00:17:25,960 --> 00:17:29,280 Speaker 1: going on in the frame so that you could tell 285 00:17:29,320 --> 00:17:33,160 Speaker 1: exactly how it's moving through that three dimensional space. Yeah, 286 00:17:33,240 --> 00:17:37,000 Speaker 1: it seems like um. In order to capture the correct perspective, 287 00:17:37,359 --> 00:17:40,159 Speaker 1: you need that additional information, even though you may not 288 00:17:40,240 --> 00:17:43,879 Speaker 1: necessarily see it. UM. It helps the the animator do that, 289 00:17:44,160 --> 00:17:47,360 Speaker 1: and the optical system to allows you to work with 290 00:17:47,760 --> 00:17:51,440 Speaker 1: more than one actor um, which was not really an 291 00:17:51,440 --> 00:17:56,440 Speaker 1: option with some of the earlier systems. So in other words, 292 00:17:56,440 --> 00:17:59,879 Speaker 1: you can, although it requires more equipment, you know, just 293 00:18:00,000 --> 00:18:03,840 Speaker 1: simply out of necessity, the optical system is really affording 294 00:18:04,080 --> 00:18:08,280 Speaker 1: the animators a an opportunity to use a greater amount 295 00:18:08,320 --> 00:18:12,520 Speaker 1: of information um both you know, from the different the 296 00:18:12,600 --> 00:18:16,280 Speaker 1: different points of data they're getting from a single actor, 297 00:18:16,680 --> 00:18:20,479 Speaker 1: but from multiple actors on the set simultaneously, which enables 298 00:18:20,520 --> 00:18:24,240 Speaker 1: them to to create more complex work right and UH. 299 00:18:24,400 --> 00:18:27,919 Speaker 1: This also gives us a good example of how the 300 00:18:27,960 --> 00:18:32,720 Speaker 1: optical motion capture systems are a passive system because you 301 00:18:32,760 --> 00:18:35,159 Speaker 1: have these sensors you're wearing that are not necessarily or 302 00:18:35,160 --> 00:18:39,000 Speaker 1: not even sensors. They're they're reflective markers that you're wearing. 303 00:18:39,040 --> 00:18:42,880 Speaker 1: They aren't connected to any sort of electronic components at all, 304 00:18:43,320 --> 00:18:47,960 Speaker 1: versus the active systems like the electromagnetic one, where you 305 00:18:48,040 --> 00:18:52,560 Speaker 1: are generating data by moving through a magnetic field and 306 00:18:52,600 --> 00:18:55,600 Speaker 1: you have these big cables attached to it. Uh. With 307 00:18:55,640 --> 00:18:58,520 Speaker 1: the optical motion capture systems. Another thing that's kind of interesting, 308 00:18:58,600 --> 00:19:00,880 Speaker 1: I think is that a lot of least the early ones, 309 00:19:01,320 --> 00:19:06,520 Speaker 1: the cameras would have infrared l ed s uh so admitters, 310 00:19:06,560 --> 00:19:10,959 Speaker 1: really that we're emitting infrared lights. That's outside our our 311 00:19:11,040 --> 00:19:15,160 Speaker 1: visible spectrum. We cannot see infrared light. But by putting 312 00:19:15,200 --> 00:19:18,320 Speaker 1: an infrared filter on the camera, you could have the 313 00:19:18,359 --> 00:19:21,879 Speaker 1: camera pick up reflections of infrared light. And that was 314 00:19:21,920 --> 00:19:25,879 Speaker 1: a way of helping to identify the sensors that you 315 00:19:25,960 --> 00:19:29,240 Speaker 1: had put on the actor. The actors, the sensors would 316 00:19:29,280 --> 00:19:33,240 Speaker 1: be reflective specifically so that the infrared light would reflect 317 00:19:33,280 --> 00:19:37,080 Speaker 1: back toward the camera and give the most accurate rendering 318 00:19:37,160 --> 00:19:41,000 Speaker 1: of what's going on at any given moment within a scene. 319 00:19:41,560 --> 00:19:45,520 Speaker 1: So um, yeah, it's another way of making sure that 320 00:19:45,680 --> 00:19:48,560 Speaker 1: the data being captured is as precise as possible. I 321 00:19:48,560 --> 00:19:50,200 Speaker 1: mean that is, of course, the goal is to try 322 00:19:50,200 --> 00:19:55,240 Speaker 1: and recreate the physical movements as truthfully as you possibly 323 00:19:55,320 --> 00:19:59,880 Speaker 1: can given all the limitations involved. Yeah, and if you're 324 00:20:00,000 --> 00:20:04,600 Speaker 1: looking for a real life easy to find an example 325 00:20:04,680 --> 00:20:08,679 Speaker 1: of this, you would look no farther than your local 326 00:20:08,760 --> 00:20:13,879 Speaker 1: video game store. Um, because the Xbox Connect h uses 327 00:20:14,000 --> 00:20:18,119 Speaker 1: very much that that exact uh form of technology. It 328 00:20:18,320 --> 00:20:21,480 Speaker 1: is using an infrared emitter, um, and it has cameras 329 00:20:21,520 --> 00:20:24,240 Speaker 1: that it uses to pick it up uh, the information 330 00:20:24,440 --> 00:20:27,040 Speaker 1: pick the information up that is coming back from what 331 00:20:27,200 --> 00:20:29,720 Speaker 1: is being reflected around the room, and anybody who who 332 00:20:29,760 --> 00:20:33,080 Speaker 1: has one is also aware that lighting is very much 333 00:20:33,119 --> 00:20:36,040 Speaker 1: an issue. Um. The way that were room is let 334 00:20:36,119 --> 00:20:39,160 Speaker 1: affects the information that the Connect is able to refer 335 00:20:39,200 --> 00:20:42,200 Speaker 1: to the Xbox. Now, it's not, while it is sophisticated, 336 00:20:42,280 --> 00:20:44,399 Speaker 1: is not as sophisticated as the kind of equipment that 337 00:20:44,440 --> 00:20:47,159 Speaker 1: they might use in making a movie or making a 338 00:20:47,280 --> 00:20:51,000 Speaker 1: video game. But it is very very similar technology, and 339 00:20:51,000 --> 00:20:53,960 Speaker 1: in some ways I would argue that it's more sophisticated 340 00:20:53,960 --> 00:20:57,080 Speaker 1: than some of those early UH systems simply because it 341 00:20:57,160 --> 00:21:00,800 Speaker 1: is able to capture a lot of information uh, Whereas 342 00:21:00,880 --> 00:21:03,560 Speaker 1: you know, the very early optical systems were only using 343 00:21:03,760 --> 00:21:08,480 Speaker 1: a handful of data points. So um, it's it's a 344 00:21:08,520 --> 00:21:11,800 Speaker 1: pretty neat device. Um, you know, not only used for gaming. 345 00:21:11,800 --> 00:21:13,679 Speaker 1: Now the hacker community has fallen in love with it 346 00:21:13,720 --> 00:21:16,280 Speaker 1: too because it can do so much and can be 347 00:21:16,359 --> 00:21:20,000 Speaker 1: used for so many things and is you know, fairly inexpensive. Yeah. 348 00:21:20,000 --> 00:21:22,680 Speaker 1: The cool thing about the Connect is that rather than 349 00:21:22,720 --> 00:21:25,480 Speaker 1: have to obviously, if you've if you've ever played an 350 00:21:25,640 --> 00:21:27,959 Speaker 1: Xbox with the connect, you know, you don't have to 351 00:21:28,040 --> 00:21:31,000 Speaker 1: go out and buy a snug body suit covered in 352 00:21:31,040 --> 00:21:33,800 Speaker 1: reflective markers in order to play. I mean, it doesn't hurt, 353 00:21:34,600 --> 00:21:36,720 Speaker 1: but uh, you know, if you're if you can pull 354 00:21:36,760 --> 00:21:39,000 Speaker 1: that look off. There a very few of us who can. 355 00:21:39,280 --> 00:21:42,560 Speaker 1: I count myself among them. But you don't have to 356 00:21:42,600 --> 00:21:45,080 Speaker 1: do that because what it's doing is it's actually projecting 357 00:21:45,200 --> 00:21:50,120 Speaker 1: essentially a grid, uh in infrared light, so you can't 358 00:21:50,160 --> 00:21:52,880 Speaker 1: see the grid, but it's being projected into the room. 359 00:21:53,040 --> 00:21:56,760 Speaker 1: And then when you move uh within the space, you 360 00:21:56,800 --> 00:21:59,440 Speaker 1: are deforming that grid. You know, the camera that's picking 361 00:21:59,520 --> 00:22:04,560 Speaker 1: up the the reflections of that infrared light can detect 362 00:22:04,600 --> 00:22:08,359 Speaker 1: when the grid's being deformed by a physical object interrupting 363 00:22:08,400 --> 00:22:11,440 Speaker 1: the grid. So as you move, you interrupt different parts 364 00:22:11,480 --> 00:22:13,800 Speaker 1: of the grid, and it can start to interpret those 365 00:22:13,880 --> 00:22:19,600 Speaker 1: as motions and commands. It's not, uh, it's not as 366 00:22:19,640 --> 00:22:22,800 Speaker 1: precise as what we're talking about with the optical systems 367 00:22:22,840 --> 00:22:25,560 Speaker 1: that are used in movies and video games, uh, to 368 00:22:25,560 --> 00:22:29,200 Speaker 1: to create them, that is, not to to play them. Um, 369 00:22:29,240 --> 00:22:32,200 Speaker 1: it's not as precise as those, but it also has 370 00:22:32,280 --> 00:22:35,520 Speaker 1: other elements that help balance it out, Like it has 371 00:22:36,200 --> 00:22:42,120 Speaker 1: regular optical cameras that can have some other software that 372 00:22:42,520 --> 00:22:46,640 Speaker 1: aids it in recognizing things like facial recognition software, which 373 00:22:46,640 --> 00:22:50,960 Speaker 1: does not necessarily rely upon that infrared grid. It relies 374 00:22:51,000 --> 00:22:55,680 Speaker 1: more on the traditional camera functions, but has the software 375 00:22:55,720 --> 00:23:00,280 Speaker 1: included that, let's the the programs within recognize who is 376 00:23:00,320 --> 00:23:04,280 Speaker 1: standing in front of it, so that combination, uh increases 377 00:23:04,320 --> 00:23:06,560 Speaker 1: the precision, which of course is very important whenever you're 378 00:23:06,560 --> 00:23:09,120 Speaker 1: playing a game. I mean, anyone who's played any sort 379 00:23:09,200 --> 00:23:12,560 Speaker 1: of game where you're using a faulty controller, or it's 380 00:23:12,600 --> 00:23:16,479 Speaker 1: just a system that hasn't been fully uh it's not 381 00:23:16,520 --> 00:23:19,080 Speaker 1: finished yet, it's just in prototype stage or whatever, you 382 00:23:19,119 --> 00:23:22,359 Speaker 1: may have noticed that it could be very frustrating to 383 00:23:22,480 --> 00:23:26,399 Speaker 1: try and control something where the actual controller is not 384 00:23:27,440 --> 00:23:31,239 Speaker 1: as responsive as you would hope. It's um not a 385 00:23:31,280 --> 00:23:34,880 Speaker 1: fun experience. But anyway, that is kind of related to 386 00:23:34,960 --> 00:23:41,560 Speaker 1: this whole motion capture technology. UM, I'm sorry what you 387 00:23:41,320 --> 00:23:43,639 Speaker 1: were You look like you have something to say, Well, no, 388 00:23:43,760 --> 00:23:46,320 Speaker 1: I was, I was going to say that. Um, you know, 389 00:23:46,359 --> 00:23:51,240 Speaker 1: we really hadn't other than my earlier statement about sports. UM, 390 00:23:51,240 --> 00:23:52,879 Speaker 1: you know, we've we've been talking about it in an 391 00:23:53,040 --> 00:23:59,120 Speaker 1: entertain amount entertainment about the the ability to capture motion 392 00:23:59,200 --> 00:24:02,800 Speaker 1: to make care act is more realistic. And um, that 393 00:24:02,800 --> 00:24:05,479 Speaker 1: that is exactly what they want to do when they 394 00:24:05,480 --> 00:24:08,399 Speaker 1: are using this in sports medicine. UM. Jonathan alluded to 395 00:24:08,480 --> 00:24:13,119 Speaker 1: earlier the difficulty in UH and capturing all the little 396 00:24:13,160 --> 00:24:18,200 Speaker 1: subtle motions that go into UM into a Major League 397 00:24:18,200 --> 00:24:22,080 Speaker 1: baseball players pitching. UM. And you know when somebody, when 398 00:24:22,080 --> 00:24:27,359 Speaker 1: somebody gets hurt, UM, sometimes they go through uh extensive surgery. 399 00:24:27,920 --> 00:24:31,399 Speaker 1: The Tommy John procedures is UH famous. You know, they 400 00:24:31,600 --> 00:24:35,720 Speaker 1: do a ligament transplant to to help rebuild a picture's elbow, 401 00:24:36,119 --> 00:24:39,119 Speaker 1: and that can really throw off, um, the mechanics of 402 00:24:39,119 --> 00:24:42,680 Speaker 1: a pictures motion. So they use this motion capture technology 403 00:24:42,680 --> 00:24:47,280 Speaker 1: to really get an idea of how, UM, how that 404 00:24:47,359 --> 00:24:50,520 Speaker 1: person is is throwing going about the mechanics of their 405 00:24:50,560 --> 00:24:53,560 Speaker 1: typical game play. And and that's exactly the same kind 406 00:24:53,600 --> 00:24:55,440 Speaker 1: of thing that they're doing when they create these very 407 00:24:55,840 --> 00:24:59,960 Speaker 1: realistic sports games. UM. But you know, in this case, 408 00:25:00,000 --> 00:25:02,680 Speaker 1: they're using it for sports medicine to see if they can, UH, 409 00:25:02,720 --> 00:25:06,439 Speaker 1: they can go back and recreate some of the motions 410 00:25:06,440 --> 00:25:10,400 Speaker 1: that made them so successful before they were injured. Now, UM, ironically, 411 00:25:10,600 --> 00:25:17,560 Speaker 1: in in UH entertainment purposes, especially video, UM, you can 412 00:25:17,720 --> 00:25:22,400 Speaker 1: get too realistic. UM. The Japanese professor massa Hiro Mori 413 00:25:22,600 --> 00:25:26,800 Speaker 1: is famous for his Uncanny Valley UM, which has been 414 00:25:26,920 --> 00:25:31,760 Speaker 1: used in uses a robotics term for a robot that 415 00:25:31,800 --> 00:25:35,080 Speaker 1: looks so much and moves so much like a human 416 00:25:35,520 --> 00:25:39,000 Speaker 1: that it it creeps us out. It looks a little 417 00:25:39,200 --> 00:25:41,919 Speaker 1: too realistic. And I can think of we're actually recording 418 00:25:41,920 --> 00:25:45,320 Speaker 1: this in December of and um. One of the movies 419 00:25:45,359 --> 00:25:46,960 Speaker 1: that comes on about this time of year is The 420 00:25:47,000 --> 00:25:54,480 Speaker 1: Polar Express, which is known, loved and reviled both for 421 00:25:54,560 --> 00:25:57,560 Speaker 1: its story and it's um and the way that they 422 00:25:57,640 --> 00:25:59,720 Speaker 1: use motion capture because the characters and there are so 423 00:25:59,800 --> 00:26:02,760 Speaker 1: realistic they're downright creepy. Yeah, it's it's one of those 424 00:26:02,760 --> 00:26:07,560 Speaker 1: things where they are almost but not quite able to 425 00:26:07,640 --> 00:26:12,359 Speaker 1: pass for a real person, so that there's just enough 426 00:26:12,640 --> 00:26:17,680 Speaker 1: off about them to be unsettling. Now, this does bring 427 00:26:17,760 --> 00:26:20,200 Speaker 1: up something else that's kind of interesting. We have an 428 00:26:20,280 --> 00:26:25,000 Speaker 1: article on how stuff works dot com about motion scan technology, 429 00:26:25,080 --> 00:26:28,800 Speaker 1: which is, as I said earlier, a proprietary technology. It's 430 00:26:29,480 --> 00:26:33,840 Speaker 1: it's more specific than just motion capture. It's specifically meant 431 00:26:34,080 --> 00:26:40,240 Speaker 1: to capture facial motion activity. So when an actor is speaking, 432 00:26:40,280 --> 00:26:44,200 Speaker 1: when they're delivering lines, the way that they furrow their 433 00:26:44,280 --> 00:26:49,760 Speaker 1: brow or move their eyes or smile, or they give 434 00:26:49,760 --> 00:26:54,280 Speaker 1: a facial take, anything like that. This system is designed 435 00:26:54,320 --> 00:26:57,520 Speaker 1: to pick that up so that it can be recreated 436 00:26:57,760 --> 00:27:01,200 Speaker 1: virtually in a game, and it was used too great effect, 437 00:27:01,240 --> 00:27:06,480 Speaker 1: in my opinion, in L A Noir. Le Noir was 438 00:27:06,600 --> 00:27:08,600 Speaker 1: a video game that came out in two thousand eleven, 439 00:27:08,760 --> 00:27:12,200 Speaker 1: and it was a game in which you played a well, 440 00:27:12,280 --> 00:27:14,520 Speaker 1: you played a couple of different characters, but the one 441 00:27:14,600 --> 00:27:18,680 Speaker 1: you played for most of the game spoiler alert was 442 00:27:18,680 --> 00:27:22,200 Speaker 1: was a a police detective. And you're kind of rising 443 00:27:22,240 --> 00:27:26,359 Speaker 1: through the ranks uh in L A U during the 444 00:27:26,600 --> 00:27:33,320 Speaker 1: uh early part of the twentieth century. And it's it's um, 445 00:27:33,359 --> 00:27:38,640 Speaker 1: it's notable in that you are, uh, you're spending most 446 00:27:38,680 --> 00:27:42,679 Speaker 1: of the game looking at people's reactions. You know. The 447 00:27:42,760 --> 00:27:45,800 Speaker 1: idea behind L A Noir It was a new type 448 00:27:45,800 --> 00:27:51,199 Speaker 1: of video game where you would interrogate characters throughout your investigations, 449 00:27:51,480 --> 00:27:54,199 Speaker 1: and as you interrogate them, you had to watch the 450 00:27:54,320 --> 00:27:58,239 Speaker 1: characters facial reactions to kind of get an idea of 451 00:27:58,280 --> 00:28:00,720 Speaker 1: whether the character was trying to be evade se or 452 00:28:00,760 --> 00:28:03,520 Speaker 1: if they were telling the truth. And you would do 453 00:28:03,600 --> 00:28:06,359 Speaker 1: things like watch for their eyes and if they weren't 454 00:28:06,359 --> 00:28:09,439 Speaker 1: able to maintain eye contact, that was an indication that 455 00:28:09,480 --> 00:28:13,119 Speaker 1: perhaps they were being less than truthful. Or if they would, 456 00:28:13,359 --> 00:28:16,320 Speaker 1: you know, twitch their mouth or clench their jaw, these 457 00:28:16,359 --> 00:28:20,800 Speaker 1: would be little little hints that perhaps there's more going 458 00:28:20,920 --> 00:28:25,600 Speaker 1: on than what they're letting onto. And obviously, if your 459 00:28:25,640 --> 00:28:29,240 Speaker 1: gameplay depends upon trying to determine whether or not a 460 00:28:29,440 --> 00:28:32,359 Speaker 1: virtual character is telling the truth, you have to be 461 00:28:32,400 --> 00:28:38,400 Speaker 1: able to represent those facial expressions as closely to reality 462 00:28:38,440 --> 00:28:42,000 Speaker 1: as possible, or else the game does not work. So 463 00:28:42,200 --> 00:28:45,400 Speaker 1: they used this motion scan technology and the way that 464 00:28:45,640 --> 00:28:49,760 Speaker 1: they did this was that they had a very brightly 465 00:28:49,920 --> 00:28:54,520 Speaker 1: lit studio that had lights trained on an actor from 466 00:28:54,560 --> 00:28:57,160 Speaker 1: just about every angle and the purpose of that was 467 00:28:57,200 --> 00:29:00,320 Speaker 1: to try and eliminate shadows, because any sort of shadows 468 00:29:00,320 --> 00:29:03,280 Speaker 1: you would have there would of course affect the actual capture. 469 00:29:03,960 --> 00:29:06,760 Speaker 1: It was really all about the light. And they used 470 00:29:07,160 --> 00:29:11,760 Speaker 1: thirty two high definition cameras. So think about that, thirty 471 00:29:11,760 --> 00:29:15,560 Speaker 1: two high definition cameras just to capture and actor's facial 472 00:29:15,680 --> 00:29:18,760 Speaker 1: performance like that's it. There, there's no other movement. The 473 00:29:18,920 --> 00:29:22,680 Speaker 1: actor is seated at the time and um and had 474 00:29:22,720 --> 00:29:26,120 Speaker 1: to remain as still as possible and just do all 475 00:29:26,160 --> 00:29:29,880 Speaker 1: the acting with their face, which for anyone out there 476 00:29:29,880 --> 00:29:34,560 Speaker 1: who's done any sort of acting, you know, that's incredibly 477 00:29:34,680 --> 00:29:39,200 Speaker 1: challenging because actors are trained to use their whole body 478 00:29:39,640 --> 00:29:42,280 Speaker 1: when they are performance making a performance. They're trained to 479 00:29:42,760 --> 00:29:46,000 Speaker 1: to really think about movement. I mean, if you're if 480 00:29:46,040 --> 00:29:48,880 Speaker 1: you're really serious about acting, you've probably taken movement classes. 481 00:29:49,240 --> 00:29:51,920 Speaker 1: And to suddenly have all of that taken away and 482 00:29:52,080 --> 00:29:54,960 Speaker 1: all of your acting is restricted to just your face, 483 00:29:55,480 --> 00:29:59,480 Speaker 1: it's pretty that's pretty dramatic. It's tough to do, but anyway, 484 00:29:59,520 --> 00:30:01,080 Speaker 1: that's what the actors had to do. They had to 485 00:30:01,080 --> 00:30:05,120 Speaker 1: sit down and restrict their acting to just their facial 486 00:30:05,960 --> 00:30:09,880 Speaker 1: expressions without it going like over the top crazy, because 487 00:30:09,920 --> 00:30:13,160 Speaker 1: that would be just as distracting as not enough performance 488 00:30:13,160 --> 00:30:17,400 Speaker 1: at all. And these thirty two cameras were paired up, 489 00:30:17,520 --> 00:30:20,880 Speaker 1: so sixteen pairs of cameras. There's technically there was a 490 00:30:20,920 --> 00:30:23,760 Speaker 1: thirty third camera as well that the director used to 491 00:30:23,880 --> 00:30:28,400 Speaker 1: watch the scene and give directions to the actors um 492 00:30:28,480 --> 00:30:32,080 Speaker 1: but these these pairs of cameras were trained on all 493 00:30:32,080 --> 00:30:34,880 Speaker 1: these different angles of the face in order to capture 494 00:30:35,000 --> 00:30:39,200 Speaker 1: that that performance so that in the virtual world they 495 00:30:39,200 --> 00:30:43,520 Speaker 1: could recreate it accurately, which to me is phenomenal. And 496 00:30:43,640 --> 00:30:47,720 Speaker 1: apparently the way the system works is you get that 497 00:30:47,920 --> 00:30:54,280 Speaker 1: virtual version of the person's face and head almost instantly, 498 00:30:54,720 --> 00:30:59,600 Speaker 1: which is kind of creepy but also awesome. It's it's 499 00:30:59,640 --> 00:31:02,959 Speaker 1: funny too that uh they used that many cameras in 500 00:31:03,160 --> 00:31:07,240 Speaker 1: the creation of a video game, because uh, elsewhere in 501 00:31:07,240 --> 00:31:11,640 Speaker 1: that article that notes that um Circus who was playing 502 00:31:11,760 --> 00:31:20,320 Speaker 1: Gollum um only had only had cameras on on him, 503 00:31:20,360 --> 00:31:23,080 Speaker 1: but in doing so, they were able to, uh to 504 00:31:23,960 --> 00:31:28,320 Speaker 1: create roughly, you know, ten thousand different kinds or identify 505 00:31:28,440 --> 00:31:31,160 Speaker 1: ten thousand different kinds of facial movements that they could 506 00:31:31,280 --> 00:31:37,200 Speaker 1: use in in animating the character on screen. So um, clearly, uh, 507 00:31:37,240 --> 00:31:41,600 Speaker 1: you know, this is very very high tech and painstaking 508 00:31:41,600 --> 00:31:44,760 Speaker 1: procedure to do, but in doing so they can they 509 00:31:44,800 --> 00:31:47,360 Speaker 1: can create very very realistic movements. Yeah, there's a lot 510 00:31:47,400 --> 00:31:52,240 Speaker 1: of number crunching involved, and frankly, the the part that 511 00:31:52,360 --> 00:31:56,400 Speaker 1: takes place after you've captured the data is can be 512 00:31:56,480 --> 00:31:59,840 Speaker 1: dramatically different from one case to the next. In some cases, 513 00:32:00,120 --> 00:32:06,200 Speaker 1: may have already created uh an animated figure pretty much 514 00:32:06,240 --> 00:32:08,480 Speaker 1: from start to finish, you might not have completely put 515 00:32:08,560 --> 00:32:13,000 Speaker 1: textures on it or or something. But you might have 516 00:32:13,400 --> 00:32:16,480 Speaker 1: essentially the way the character is going to look in 517 00:32:16,520 --> 00:32:21,000 Speaker 1: the finished product, uh, and then you just map it 518 00:32:21,080 --> 00:32:23,800 Speaker 1: to the movements that you've captured and it's and there 519 00:32:23,800 --> 00:32:26,720 Speaker 1: it goes. And in other cases you might see that 520 00:32:26,880 --> 00:32:29,600 Speaker 1: what they do is they capture the motions and then 521 00:32:29,640 --> 00:32:33,560 Speaker 1: you essentially have what looks like a very primitive stick 522 00:32:33,600 --> 00:32:37,200 Speaker 1: figure skeleton that moves in the way that the actor moved, 523 00:32:37,520 --> 00:32:41,040 Speaker 1: but there's no definition, there's no character there yet. And 524 00:32:41,520 --> 00:32:44,480 Speaker 1: you may have animators who build the character somewhat based 525 00:32:44,560 --> 00:32:47,280 Speaker 1: upon the way the actor moved through the space, so 526 00:32:47,360 --> 00:32:51,040 Speaker 1: that perhaps the character's design is not finalized until you've 527 00:32:51,080 --> 00:32:54,479 Speaker 1: captured that that performance, and the performance helps guide the 528 00:32:54,520 --> 00:32:58,680 Speaker 1: design of the character. It all depends on the specific 529 00:32:58,720 --> 00:33:03,080 Speaker 1: technology that's being you and the preference of the crew 530 00:33:03,200 --> 00:33:05,880 Speaker 1: that's that's designing whatever it is that they're making, whether 531 00:33:05,920 --> 00:33:08,880 Speaker 1: it's a video game or movie, TV show, commercial, whatever 532 00:33:08,920 --> 00:33:12,840 Speaker 1: it happens to be. UH. In the case of digital puppetry, 533 00:33:12,880 --> 00:33:16,720 Speaker 1: obviously you would already have the the full character realized, 534 00:33:16,880 --> 00:33:21,280 Speaker 1: so that just by using whatever control mechanism happens to 535 00:33:21,360 --> 00:33:25,000 Speaker 1: be there, you would be able to make the puppet 536 00:33:25,000 --> 00:33:29,320 Speaker 1: move in real time, otherwise it's not really puppetry. Um. 537 00:33:29,400 --> 00:33:31,440 Speaker 1: And again that's sort of like the if you've been 538 00:33:31,440 --> 00:33:33,880 Speaker 1: to that that turtle talk thing I talked about, the 539 00:33:34,760 --> 00:33:38,400 Speaker 1: Disney World or Disneyland. Um. I'm sure there are other 540 00:33:38,560 --> 00:33:42,040 Speaker 1: similar ones. I think Monsters Inc. Laugh Factory has a 541 00:33:42,080 --> 00:33:45,720 Speaker 1: similar setup where you've got a digital character on a 542 00:33:45,800 --> 00:33:49,120 Speaker 1: screen that can react in real time to things that 543 00:33:49,120 --> 00:33:52,600 Speaker 1: are happening within the physical environment. So they interact with 544 00:33:52,600 --> 00:33:55,640 Speaker 1: the audience like they'll specifically single people out and chat 545 00:33:55,720 --> 00:33:59,680 Speaker 1: with people in the audience. And Um, to two kids, 546 00:33:59,720 --> 00:34:02,360 Speaker 1: this is amazing. I means the cartoon character acting in 547 00:34:02,440 --> 00:34:05,840 Speaker 1: real time, it's a real person. Now, Uh, two adults, 548 00:34:05,840 --> 00:34:10,719 Speaker 1: it's fascinating because they're like, how the heck did that happen? Um, 549 00:34:10,760 --> 00:34:13,319 Speaker 1: But yeah, that's it's all based on this same sort 550 00:34:13,360 --> 00:34:17,160 Speaker 1: of technology. UM. And It's it's really interesting to me 551 00:34:17,360 --> 00:34:21,880 Speaker 1: to see how the field is evolving over time, because 552 00:34:21,880 --> 00:34:25,120 Speaker 1: things like the connect show that we are adapting the 553 00:34:25,239 --> 00:34:27,880 Speaker 1: same sort of technology in different ways. We're using different 554 00:34:27,880 --> 00:34:31,840 Speaker 1: implementations to essentially do the same thing, and that perhaps 555 00:34:32,000 --> 00:34:35,560 Speaker 1: we will get to a point where we won't have 556 00:34:35,680 --> 00:34:40,239 Speaker 1: to worry about all the sensors so much. Um, you 557 00:34:40,280 --> 00:34:44,799 Speaker 1: can maybe have an actor who's not completely coded and 558 00:34:44,880 --> 00:34:48,560 Speaker 1: stickers perform and and you could capture all that data 559 00:34:48,680 --> 00:34:52,240 Speaker 1: without having to worry about, you know, tracking these little dots. 560 00:34:52,719 --> 00:34:54,520 Speaker 1: That might be something that we've seen in the future. 561 00:34:54,560 --> 00:34:56,080 Speaker 1: I mean the motion scan is kind of like that 562 00:34:56,120 --> 00:35:03,400 Speaker 1: because before motion scan with that facial acting uh technology. Uh, 563 00:35:03,440 --> 00:35:07,200 Speaker 1: whenever I saw anyone who was having their face tracked 564 00:35:07,239 --> 00:35:11,040 Speaker 1: for a performance, they always were wearing those tiny little 565 00:35:11,120 --> 00:35:14,200 Speaker 1: white stickers all over their face to track. I mean, 566 00:35:14,200 --> 00:35:15,759 Speaker 1: we've got a lot of muscles in our face. There's 567 00:35:15,760 --> 00:35:18,320 Speaker 1: something like nineteen muscles or something that you have to track, 568 00:35:18,400 --> 00:35:21,440 Speaker 1: so um, you would have all these little dots on 569 00:35:21,480 --> 00:35:23,600 Speaker 1: your face to track those motions. Well, with motion scan 570 00:35:23,680 --> 00:35:27,200 Speaker 1: you don't need those anymore. So maybe we'll see something 571 00:35:27,239 --> 00:35:30,240 Speaker 1: like that. Of course, that would really depend upon perhaps 572 00:35:30,320 --> 00:35:34,560 Speaker 1: the lighting, which could if you're shooting a virtual character 573 00:35:34,640 --> 00:35:36,600 Speaker 1: that's next to real characters like in The Lord of 574 00:35:36,640 --> 00:35:40,560 Speaker 1: the Rings, real being I guess you know, your mileage 575 00:35:40,600 --> 00:35:43,680 Speaker 1: may very I mean they're hobbits, but anyway, when you're 576 00:35:43,719 --> 00:35:46,239 Speaker 1: next to real people, clearly you can't mess with the 577 00:35:46,360 --> 00:35:48,960 Speaker 1: lighting too much or it'll just make the whole scene 578 00:35:49,000 --> 00:35:54,760 Speaker 1: look strange. Speaking of strange, UM, well, you might think 579 00:35:54,840 --> 00:36:00,640 Speaker 1: that the techniques used in motion capture uh, um, you know, 580 00:36:00,680 --> 00:36:03,239 Speaker 1: bringing film into a uh you know, adding a lot 581 00:36:03,239 --> 00:36:08,000 Speaker 1: of advancement to to film. Um basically uh, some people 582 00:36:08,120 --> 00:36:12,720 Speaker 1: sort of regardless as cheating. Yeah. I did some research 583 00:36:12,840 --> 00:36:17,319 Speaker 1: that that indicated that, um, although some other types of 584 00:36:17,360 --> 00:36:22,320 Speaker 1: animation are considered you know, considered more artful, UM, motion 585 00:36:22,360 --> 00:36:26,840 Speaker 1: capture is sort of not everyone. But some people say, well, 586 00:36:26,880 --> 00:36:29,200 Speaker 1: you know it's it's not oscar worthy because you were 587 00:36:29,280 --> 00:36:34,840 Speaker 1: using these computer add animation techniques that that really, um 588 00:36:35,520 --> 00:36:38,600 Speaker 1: are simulating human motion, and it's just it's just not real. 589 00:36:39,080 --> 00:36:42,160 Speaker 1: And uh. The argument that I've seen used against it is, well, 590 00:36:42,320 --> 00:36:47,560 Speaker 1: you consider rotoscoping, okay, why don't you consider motion capture, 591 00:36:47,560 --> 00:36:51,479 Speaker 1: which is a kind of descendant from this technology. Why 592 00:36:51,520 --> 00:36:54,360 Speaker 1: why isn't that okay to uh, you know, to consider 593 00:36:54,440 --> 00:36:59,520 Speaker 1: for um quality and and and for rewards. But um, 594 00:36:59,560 --> 00:37:03,680 Speaker 1: apparently it's a it's sort of a hot topic among um, 595 00:37:03,719 --> 00:37:07,680 Speaker 1: among movie makers. Yeah. I can see one animator a 596 00:37:07,680 --> 00:37:11,799 Speaker 1: traditional animator, or even a computer animator. I mean that's 597 00:37:12,280 --> 00:37:15,719 Speaker 1: closer and closer to becoming traditional already but either hand 598 00:37:15,800 --> 00:37:19,200 Speaker 1: drawn animation or computer animation. Someone who goes through the 599 00:37:19,200 --> 00:37:22,200 Speaker 1: trouble of animating these things and doing a lot of 600 00:37:22,200 --> 00:37:26,520 Speaker 1: this work. Uh, by hand seems like it's the wrong term, 601 00:37:26,560 --> 00:37:30,440 Speaker 1: but but personally going through and creating these performances, I 602 00:37:30,480 --> 00:37:33,520 Speaker 1: can see where they might feel that way. Um, I 603 00:37:33,560 --> 00:37:35,520 Speaker 1: have a completely different perspective on it. Of course, I'm 604 00:37:35,520 --> 00:37:37,960 Speaker 1: not an animator, so that's part of it, but I 605 00:37:38,000 --> 00:37:40,520 Speaker 1: think of it as creating a performance. And in the 606 00:37:40,560 --> 00:37:42,840 Speaker 1: sense of creating a performance, I think it's a completely 607 00:37:42,920 --> 00:37:48,880 Speaker 1: legitimate tool because you're still relying on an actor to 608 00:37:49,280 --> 00:37:54,319 Speaker 1: create a performance that that that people will relate to, 609 00:37:54,400 --> 00:37:58,160 Speaker 1: whether it's a character that you're supposed to love or 610 00:37:58,239 --> 00:38:03,800 Speaker 1: hate or fear, that all is dependent upon the animator 611 00:38:03,840 --> 00:38:08,320 Speaker 1: and the actor and several other people working to create 612 00:38:08,400 --> 00:38:13,320 Speaker 1: this this performance. And uh, I don't see anything wrong 613 00:38:13,360 --> 00:38:16,960 Speaker 1: with that. That to me is a completely legitimate form 614 00:38:17,080 --> 00:38:22,399 Speaker 1: of creating the art of entertainment. So um, I mean, 615 00:38:22,440 --> 00:38:25,839 Speaker 1: I do understand from an artistic perspective where some people 616 00:38:25,880 --> 00:38:27,560 Speaker 1: could have a problem with it. But but if you 617 00:38:27,680 --> 00:38:30,480 Speaker 1: take a bigger picture look and not not just you 618 00:38:30,520 --> 00:38:33,680 Speaker 1: know what technique you're using, but the end goal of creating, 619 00:38:34,600 --> 00:38:36,520 Speaker 1: whether you want to call it art or not, but 620 00:38:36,719 --> 00:38:40,920 Speaker 1: creating something that has an impact to the viewer or 621 00:38:41,040 --> 00:38:43,880 Speaker 1: player in the case of a video game, I think 622 00:38:43,960 --> 00:38:47,960 Speaker 1: that's more important. But then again, I'm like I said, 623 00:38:47,960 --> 00:38:50,239 Speaker 1: I'm not an animator, so I don't have that kind 624 00:38:50,280 --> 00:38:53,319 Speaker 1: of emotional attachment, you know, I'm not vested in it 625 00:38:53,360 --> 00:38:56,399 Speaker 1: in that way. So UM, I'll be curious to hear 626 00:38:56,440 --> 00:38:58,960 Speaker 1: what our listeners think if they think that there is 627 00:38:59,040 --> 00:39:03,560 Speaker 1: motion capture? Is that cheating? Is it? Uh? Is it 628 00:39:03,680 --> 00:39:07,440 Speaker 1: as Red versus Blue would have you say, a legitimate strategy? 629 00:39:07,960 --> 00:39:10,000 Speaker 1: What what do you think? What do you consider a 630 00:39:10,000 --> 00:39:13,680 Speaker 1: motion capture? You should less know? Yeah, I UM, I 631 00:39:14,040 --> 00:39:21,120 Speaker 1: do see where UM it might make a traditional animator concerned, 632 00:39:21,239 --> 00:39:25,160 Speaker 1: but I don't. I don't really think it diminishes their UM, 633 00:39:25,239 --> 00:39:29,960 Speaker 1: their artistic value, to to UM, to a work whatever 634 00:39:29,960 --> 00:39:32,400 Speaker 1: it may be that they are working on. UM. And 635 00:39:32,440 --> 00:39:35,439 Speaker 1: there are certain times I'm sure where uh you would 636 00:39:35,520 --> 00:39:39,600 Speaker 1: argue that using these techniques is completely inappropriate to what 637 00:39:40,040 --> 00:39:43,279 Speaker 1: they might do UM. But yeah, I mean it's it's 638 00:39:43,320 --> 00:39:47,680 Speaker 1: always a concern when UM you start saying, well, the 639 00:39:47,680 --> 00:39:49,800 Speaker 1: machine can do it, and we don't really need people 640 00:39:49,840 --> 00:39:53,200 Speaker 1: to do it, so get out. Yeah, I don't think 641 00:39:53,200 --> 00:39:57,319 Speaker 1: that's ever gonna be um always the fully the case, 642 00:39:57,360 --> 00:40:00,160 Speaker 1: because you're going to have certain characters within movies that 643 00:40:00,239 --> 00:40:04,480 Speaker 1: are going to be so different from the way humans 644 00:40:04,680 --> 00:40:09,160 Speaker 1: are built, so to speak, that that, uh, that motion 645 00:40:09,200 --> 00:40:12,399 Speaker 1: capture would not be practical. For example, like let's say 646 00:40:12,400 --> 00:40:17,880 Speaker 1: that the character that you're creating has really super long arms, 647 00:40:18,080 --> 00:40:20,080 Speaker 1: and you know, you've got an actor who's pretty lanky, 648 00:40:20,160 --> 00:40:22,640 Speaker 1: but but their arms are not as long as the 649 00:40:22,719 --> 00:40:25,640 Speaker 1: character's arms. Uh, if you were just to a direct 650 00:40:25,640 --> 00:40:29,799 Speaker 1: translation of the actor's movements into the animation, it might 651 00:40:29,840 --> 00:40:34,879 Speaker 1: not look right because the character has different dimensions, their 652 00:40:34,920 --> 00:40:38,720 Speaker 1: body is built differently than the actor. And so without 653 00:40:38,800 --> 00:40:41,400 Speaker 1: tweaking it, without having an animator go in there and 654 00:40:41,440 --> 00:40:45,040 Speaker 1: adjust this and make it look correct compared to what 655 00:40:45,080 --> 00:40:47,200 Speaker 1: the you know, the the vision is for the movie, 656 00:40:47,760 --> 00:40:50,600 Speaker 1: it doesn't come out correctly, it doesn't look right. So 657 00:40:51,040 --> 00:40:55,839 Speaker 1: I think there's very little risk of motion capture ever 658 00:40:56,000 --> 00:41:01,200 Speaker 1: taking that away completely. Plus, there is something to you know, 659 00:41:01,320 --> 00:41:06,279 Speaker 1: creating a performance through traditional animation that you know, it 660 00:41:06,320 --> 00:41:09,160 Speaker 1: does feel differently the motion capture, but that's not a 661 00:41:09,160 --> 00:41:13,040 Speaker 1: bad thing, Like it just depends upon the the vision 662 00:41:13,040 --> 00:41:17,000 Speaker 1: of the director and what the tone of the piece 663 00:41:17,080 --> 00:41:19,680 Speaker 1: needs to be. And in some cases motion capture is 664 00:41:19,680 --> 00:41:21,160 Speaker 1: going to be the best way to achieve that. In 665 00:41:21,200 --> 00:41:25,879 Speaker 1: other cases, motion capture would make it distracting. So um yeah, 666 00:41:25,920 --> 00:41:30,120 Speaker 1: I think as long as we maintain this desire for 667 00:41:30,640 --> 00:41:34,960 Speaker 1: different types of of entertainment and techniques, then there's not 668 00:41:35,040 --> 00:41:40,160 Speaker 1: really any risk of making one disappear. I agree completely, 669 00:41:40,200 --> 00:41:44,520 Speaker 1: I really do. It's it's just a different animals, yep, yep. 670 00:41:44,560 --> 00:41:48,080 Speaker 1: But hey, guys, if you want to chime in on 671 00:41:48,120 --> 00:41:51,480 Speaker 1: this motion capture discussion, please do. Let's know, send us 672 00:41:51,480 --> 00:41:54,439 Speaker 1: an email, are address this tech stuff at Discovery dot com, 673 00:41:54,600 --> 00:41:57,360 Speaker 1: or get in touch with us on Twitter or Facebook 674 00:41:57,640 --> 00:42:00,719 Speaker 1: our handle at both of those. It's text stuff. H. S. 675 00:42:01,000 --> 00:42:02,800 Speaker 1: W and Chris and I will talk to you again 676 00:42:03,480 --> 00:42:07,640 Speaker 1: really soon for more on this and thousands of other topics. 677 00:42:07,920 --> 00:42:14,279 Speaker 1: Is it how staff works dot com Brought to you 678 00:42:14,320 --> 00:42:16,440 Speaker 1: by the two thousand twelve Toyota Camra