Speaker 1: Technology with tech Stuff from stuff dot Com. Hey there,
Speaker 1: and welcome to Tech Stuff. I'm your host, Jonathan Strickland,
Speaker 1: podcaster extraordinaire with how stuff works. And as you guys know,
Speaker 1: I loves me some technologies and that's what I'd like
Speaker 1: to cover on this show. And in the past I've
Speaker 1: covered concepts like Moore's law, and that can tend to
Speaker 1: lead people into thinking that, because computer processing power effectively
Speaker 1: doubles every two years or so, that the entire world
Speaker 1: is going to be transformed into a crazy techno wizard
Speaker 1: world that you'd see in an imaginative science fiction film.
Speaker 1: It's gonna happen any day now, and we're all headed
Speaker 1: that way. It's a foregone conclusion. However, from a day
Speaker 1: to day perspective, it doesn't seem like we're going all
Speaker 1: that quickly, despite that amazing progress and computer process using power.
Speaker 1: And one of the reasons there are many, but one
Speaker 1: of the reasons for this is that while computer processing
Speaker 1: power tends to follow that pathway that Gordon Moore observed
Speaker 1: several decades ago, other aspects of technology aren't on quite
Speaker 1: as predictable a path, nor do they advance at such
Speaker 1: a reliable accelerated rate. And because of that, because some
Speaker 1: aspects of technology lag behind the development of processing power,
Speaker 1: we don't get that crazy future that we've been promised
Speaker 1: since say the nineteen fifties. Now, one of those aspects,
Speaker 1: one of the things that does not evolve as quickly
Speaker 1: as processing power, would be user interfaces. Now, these are
Speaker 1: the ways that we interact with our devices, such as
Speaker 1: our computers. So the typical computer user, they're usually relying
Speaker 1: upon your basic keyboard and mouse combination. It's the same
Speaker 1: thing that's been in place, say the nineteen eighties and
Speaker 1: personal computers. Xerox Park was introducing things like the mouse
Speaker 1: and the graphic user interface much earlier than that, but
Speaker 1: it was really the Apple Macintosh that brought the graphic
Speaker 1: user interface and the mouse along with the keyboard to
Speaker 1: the personal computing space. The keyboard, of course, had already existed. Uh,
Speaker 1: there's a problem with this type of interface. There's actually
Speaker 1: a couple. But one is that it requires users to
Speaker 1: be both able to manipulate the input devices, which would
Speaker 1: be the keyboard and mouse. You have to actually be
Speaker 1: able to move them in order to interact with your computer.
Speaker 1: And the other problem is that there's a learning curve.
Speaker 1: You actually have to learn how to use those devices
Speaker 1: in order to interact with your computer properly. It's not
Speaker 1: a natural way to interact with a machine, or with
Speaker 1: anything else. It's not the way we tend to interact
Speaker 1: with the stuff in our everyday world. We don't have
Speaker 1: to type out codes or move a random part of
Speaker 1: an object in order to make something else move. We
Speaker 1: are much more direct with our input whenever we are
Speaker 1: interacting with our environment. So in order to learn how
Speaker 1: to use a computer, we actually have to go through
Speaker 1: this process of learning how keyboards and computer mices work.
Speaker 1: Otherwise nothing much gets done. Now that's not to say
Speaker 1: that once the keyboard and mouse combo came out we
Speaker 1: stopped trying to find ways to interact with technology, because
Speaker 1: engineers have been working on everything from light pens to
Speaker 1: touch screens, to gesture controls to voice commands all the
Speaker 1: way back to when the keyboard and mouse started to
Speaker 1: become a thing. And this is all in an effort
Speaker 1: to broaden the way we control computers and electronics. And
Speaker 1: I've covered some of those technologies in earlier episodes of
Speaker 1: Tech Stuff, and I've explained that many of them, like
Speaker 1: voice recognition do not follow a path that's nearly as
Speaker 1: accelerated as processing power. We've gotten really good with voice
Speaker 1: recognition over the last say, five or six years, but
Speaker 1: for many years it was lagging behind other technologies. And
Speaker 1: this is the problem with making those big promises about
Speaker 1: what the future is going to be like. Often that
Speaker 1: is based on an assumption that everything is moving forward
Speaker 1: at the same speed, and that's just not true. Well,
Speaker 1: today I wanted to cover a specific type of technology
Speaker 1: that can be used in user interfaces, although that's only
Speaker 1: one application of this technology, and that is eye tracking technology.
Speaker 1: A bit later in this episode, I'm going to play
Speaker 1: an interview I had with Oscar Werner of Toby Tech,
Speaker 1: and Toby Tech is a business unit of a company,
Speaker 1: Toby that is pioneering eye tracking technology for all sorts
Speaker 1: of applications, both on the business to business side and
Speaker 1: the business to consumer side. But before I get to
Speaker 1: that interview, I thought it might be interesting to talk
Speaker 1: about the evolution of eye tracking in general, because it's
Speaker 1: actually a really cool story, and you I learned stuff
Speaker 1: I just didn't know, including stuff that schemed me out
Speaker 1: a little bit. So I want to share my skiviness
Speaker 1: with you. Wait, no, that's not right. I want to
Speaker 1: share the stuff that schemed me out with you. My
Speaker 1: skiviness is a matter for my own personal life. Back
Speaker 1: off alright, To begin with, eye tracking technology is the
Speaker 1: product of a few different disciplines, and they originate from
Speaker 1: different perspectives. You've got philosophical approaches, you have technical approaches,
Speaker 1: you have biological approaches, and you also have human behavior
Speaker 1: along with psychology and physiology. So in the eighteenth century,
Speaker 1: which I feel I need to point out, was an
Speaker 1: era that had very few personal computers in it because
Speaker 1: they had not yet been invented. There was a guy
Speaker 1: named William Porterfield, who should not be confused with professional
Speaker 1: cricket player from Ireland, totally different person. This was William
Speaker 1: Porterfield in the eighteenth century, an Englishman who made some
Speaker 1: general notes about oculomotor behavior and reading, in other words,
Speaker 1: how our eyes behave as we read text. Now, Porterfield's
Speaker 1: work was not based off empirical evidence, but rather casual
Speaker 1: observations and some hypotheses. But later scientists would study this
Speaker 1: behavior in greater detail and with more rigorous scientific approaches.
Speaker 1: So some of them looked at it from a physiological standpoint,
Speaker 1: some looked at it from a philosophical standpoint. Some of
Speaker 1: them combined the two, and a man named Louis Emil
Speaker 1: Javal conducted a study of how our eyes behave when
Speaker 1: we read text. He was working in the late nineteenth century,
Speaker 1: around the eighteen sixties and eighteen seventies. He published his
Speaker 1: work about the US in eighteen seventy eight, and he
Speaker 1: made his observations by directly watching people's eyes as they
Speaker 1: read lines of text. And he saw that when you
Speaker 1: do that, your eyes don't just glide seamlessly across a page. Instead,
Speaker 1: your eyes have lots of stops and starts. Those motions
Speaker 1: he gave names, so he called every time you stopped,
Speaker 1: when you would your eyes would pause as they went across,
Speaker 1: he called those fixations. And he would call all the
Speaker 1: bits where your eyes would move across rapidly Saccadays, those
Speaker 1: would be the starts. Now, saccad comes from a French
Speaker 1: word that was originally used to describe horse movements, and
Speaker 1: a rough translation into English might be jerk, as in
Speaker 1: the motion, not as in some jerk face who's giving
Speaker 1: you a hard time. So Javal's work was interesting in that,
Speaker 1: for the most part. The only studies in oculu motor
Speaker 1: function up to that time were limited to diagnos sing
Speaker 1: and perhaps treating dysfunction, so the work wasn't all about
Speaker 1: learning more of what typical behavior is like. In other words,
Speaker 1: doctors normally didn't think to look into the way our
Speaker 1: eyes move unless something atypical was going on with the patient.
Speaker 1: Javal started a movement in medicine and biology that, pardon
Speaker 1: the pun, shifted the focus. Other scientists observed similar similar
Speaker 1: oculomotor behavior, so this was all emerging around the same time.
Speaker 1: There are actually quite a few people who were looking
Speaker 1: into it, so to speak. Javal just became the most
Speaker 1: famous of all those people. So I don't mean to
Speaker 1: suggest he was the only one, but rather he's the
Speaker 1: one that gets most of the credit, and that is
Speaker 1: because of another person named Edmund Huey, who was a
Speaker 1: clever fellow from the late nineteenth and early twenty centuries.
Speaker 1: He made more progress in this study by creating an
Speaker 1: invention that sounds like it came straight out of a
Speaker 1: clockwork orange. It consisted of a plaster I cup that
Speaker 1: would fit over a subject's I so it would go
Speaker 1: right up against the person's eye, and the cup had
Speaker 1: a lever attached to it that went off to the side.
Speaker 1: At the other end of the lever there was a pen,
Speaker 1: and the pen could move across a rotating smoked drum,
Speaker 1: and thus the pen would make marks across the smoked
Speaker 1: surface of this drum that rotated around. And the idea
Speaker 1: was that this would allow you to look at those
Speaker 1: marks and map that to I've movements that are happening
Speaker 1: while you're actually giving the subject a test. So as
Speaker 1: the subject's eyes would scan text, the movements of those
Speaker 1: eyes would make the plaster cup move, which would transfer
Speaker 1: that motion to the lever and ultimately to the pen
Speaker 1: that was resting against the rotating drum. And this sounds
Speaker 1: absolutely ghastly to me, but it gave valuable information to
Speaker 1: Hueie about how people's eyes move when they're reading. Now,
Speaker 1: in his papers on the subject, hue would refer back
Speaker 1: to Javal's work and his research on the physiology of reading.
Speaker 1: There were other scientists who are looking into this subject
Speaker 1: as well, but Huey's work and the citation of Javel
Speaker 1: have led many to simplify history and just say they're
Speaker 1: the ones responsible for the research. On i've movements in
Speaker 1: the early days. Later on, another scientist, George Malcolm Stratton,
Speaker 1: would start to study how we move our eyes as
Speaker 1: we perceive illusions. Now, this is really interesting. Optical illusions
Speaker 1: depend upon our eyes perceiving something that is either not
Speaker 1: there or is there in a way that is different
Speaker 1: from the actual physical thing that we are perceiving, and
Speaker 1: therefore the way our eyes behave is very important to
Speaker 1: how those illusions work. What is it that's going on
Speaker 1: physiologically with our eyes, how much of that is dependent
Speaker 1: upon my behavior versus what's going on in our brains.
Speaker 1: This would slowly lead scientists to the conclusion that our
Speaker 1: eyes are really an extension of our brains, and that
Speaker 1: the process of thinking is very closely, if not synonymously,
Speaker 1: connected with the process of seeing. And I think that's
Speaker 1: pretty cool too. Stratton's work would publish in the early
Speaker 1: twentieth century and also help inform scientists and others about
Speaker 1: the nature of our eyes. All of this is to say,
Speaker 1: these early studies began to coalesce around that concept of
Speaker 1: not just how our eyes move as we perceive, but
Speaker 1: also where our focus tends to be directed. Knowing about
Speaker 1: focus is important for lots of different fields, some of
Speaker 1: which appear to be completely independent of technology and user interfaces.
Speaker 1: For example, let's say that you are a graphics designer
Speaker 1: and you have a job. You're given a contract, and
Speaker 1: your job is to create a sign that will warn
Speaker 1: people to potential danger. Maybe it's a warning sign for
Speaker 1: let's say radioactive material, and you want to make sure
Speaker 1: you get across to people that anything beyond that sign
Speaker 1: could pose as a threat to that person's health. There
Speaker 1: are many things you would have to consider before you
Speaker 1: start making that sign. For example, which colors should you use.
Speaker 1: You're gonna want something that's going to have a good
Speaker 1: level of contrast, so that whatever the message is that
Speaker 1: you're trying to convey will be easy for people to perceive,
Speaker 1: even if people might have some form of color blindness.
Speaker 1: Then you have to ask, all right, well, if there
Speaker 1: are any words on there, what typeface should you use?
Speaker 1: I mean, it needs to be easily legible, and where
Speaker 1: do you draw the focus on this sign so that
Speaker 1: the message you are sending gets to your audience and
Speaker 1: they actually get the point. Being able to track eye
Speaker 1: movement gives us more information about how we humans work,
Speaker 1: not just how we can interact with stuff. As for
Speaker 1: eye tracking technology, it thankfully evolved beyond the need for
Speaker 1: eye cups and mechanical levers. Otherwise I would just be
Speaker 1: running around screaming inside of the studio because I have
Speaker 1: a thing about eyes. But one methodology still relied on
Speaker 1: the subject wearing special equipment, and it was a system
Speaker 1: consisting of a pair of contact lenses with a sort
Speaker 1: of special feature, which is dependent upon the actual implementation
Speaker 1: of the technology. Some versions would use contact lenses that
Speaker 1: had a magnetic field sensor connected to them, or they
Speaker 1: might have an embedded mirror that would reflect light, and
Speaker 1: then you would have an external system that has partnered
Speaker 1: with these contact lenses and that would help monitor the
Speaker 1: movement of the special component of those contact lenses in
Speaker 1: turn that would track eye motion. This typically would provide
Speaker 1: a really accurate indication of what the eye is doing
Speaker 1: at any given time, assuming that the contact lens fits
Speaker 1: properly and doesn't itself slide a bit across the surface
Speaker 1: of the eye. Obviously, if that happens, then it's messing
Speaker 1: up your metrics because you're looking at the movement of
Speaker 1: the contact lens, not necessarily the movement of the eye itself.
Speaker 1: But for something like a computer or a mobile device
Speaker 1: that a consumer might purchase and you want an interface
Speaker 1: for that, you probably don't want to deal with putting
Speaker 1: in special contacts. So the interfaces that we're finding with
Speaker 1: personal electronics and computers typically rely upon optical tracking. And
Speaker 1: that means exactly what you think. It means. You're using
Speaker 1: some form of camera to monitor where you are looking.
Speaker 1: The device uses cameras that can detect your eyes using
Speaker 1: software that has image recognition algorithms running in it, So
Speaker 1: it's looking for different aspects of eyes that are common
Speaker 1: across many people. And once it identifies, all right, that's
Speaker 1: an eye, then it starts to try and track where
Speaker 1: the eye is looking. Typically, you have to end up
Speaker 1: using a system to set a baseline for this. You
Speaker 1: can't just turn it on and start looking. You have
Speaker 1: to calibrate it. So you calibrate the system so it
Speaker 1: knows where you're looking. It might guide you into looking
Speaker 1: at specific regions of a screen, whether it's a mobile
Speaker 1: device or a computer or whatever it might be. And
Speaker 1: that way, once it knows quote unquote that you are
Speaker 1: looking where you're supposed to, it can then uh interpret
Speaker 1: where your eye is looking from that point forward, because
Speaker 1: it's calibrated from a known set of standards. And from
Speaker 1: that point forward, assuming that everything is working properly, every
Speaker 1: time you look at some other part of the screen,
Speaker 1: the eye tracking software can track your gaze and know
Speaker 1: exactly where you are looking. That's the basic idea behind it. Now,
Speaker 1: when we listen to in on an interview in just
Speaker 1: a few minutes, Mr Werner will explain how Toby's technology
Speaker 1: accomplishes this. In particular, they have a specific kind of
Speaker 1: approach to using cameras to track eye movements. It's not
Speaker 1: that different from the way motion sensors like the Microsoft
Speaker 1: Connect work, only obviously, eye tracking technology is more precisely
Speaker 1: tuned to look at the movement of your eyeballs, not
Speaker 1: your overall body. Now, this technology allows for all sorts
Speaker 1: of potential applications, and some of them are passive, which
Speaker 1: means that people can use the eye tracking data to
Speaker 1: get an idea of where people direct their focus for
Speaker 1: any given situation. So that might be when someone is
Speaker 1: looking at an advertisement, or looking at a web page,
Speaker 1: or playing a game. It can be more active. It
Speaker 1: doesn't have to be a passive implementation, it can allow
Speaker 1: people the chance to send input into a computer by
Speaker 1: staring at specific parts of a screen. This is useful
Speaker 1: for those people who may otherwise be unable to move.
Speaker 1: They can use these kind of interfaces in order to communicate.
Speaker 1: And there are countless other applications that exist now and
Speaker 1: even more that we haven't even thought of yet. To
Speaker 1: get a better idea of what this tech is all
Speaker 1: about and how it might be used in the future,
Speaker 1: I talked with Oscar Warner. He's the business unit president
Speaker 1: of Toby Tech, and we'll hear that interview right after
Speaker 1: we take this quick break to thank our sponsor. Let's
Speaker 1: serve by you introducing yourself and your title and the company.
Speaker 1: So my name is Oscar Warner. I'm president of Toby Tech,
Speaker 1: and Toby Tech is basically one of three business units
Speaker 1: in Toby U in Toby Group, so um Toby there
Speaker 1: are three business units. Toby Dinox is providing solutions for
Speaker 1: people with disabilities. Someone with a S who cannot move,
Speaker 1: any who cannot move at all can basically communicate via
Speaker 1: looking at a computer or send emails via just looking
Speaker 1: at the computer, or somebody with CP who is pastic
Speaker 1: can type emails and write on a thesis or or
Speaker 1: or or write on a computer and use a computer
Speaker 1: just with the rice. And then it's Toby Toby Pro
Speaker 1: which is focusing on selling research people's solutions to to
Speaker 1: universities and to companies like Proper and gaming the U
Speaker 1: in the level to understand human behavior. And then it's
Speaker 1: Toby Tech who is the developer of all the core
Speaker 1: eye tracking technology and is selling this to the consumer markets,
Speaker 1: which may be PIEC, gaming, the PC industry, smartphone industry,
Speaker 1: v R, a R industry, so selling to the big
Speaker 1: big o emsum of which you know pretty much all
Speaker 1: the names. Right, So let's start just by kind of
Speaker 1: explaining what I track tracking technology does in general, so
Speaker 1: without getting into the mechanics of it, what exactly is
Speaker 1: its purpose as far as as a technology goes. So,
Speaker 1: eye tracking does two things, um um. It basically makes
Speaker 1: technology understand you as a human being, and it does
Speaker 1: that in two different ways. You know. The first one
Speaker 1: is what we in already call insight. It makes devices
Speaker 1: know what you're paying attention to, and when you think
Speaker 1: about it, what you're paying attention to is your interest
Speaker 1: or your intent at that given point in time. So
Speaker 1: a computer with eye tracking know what your intention is
Speaker 1: and when you think about it is if you think
Speaker 1: on your in front of computer, you're looking at an
Speaker 1: icon that you want to click on. The first thing
Speaker 1: you do is you look at it. The second thing
Speaker 1: you do is you take your mouse and drag the
Speaker 1: mouse and throw over it. And the third thing you
Speaker 1: do is you click on your mouth or in your
Speaker 1: touchpad or in your control or whatever. But a computer
Speaker 1: with an eye tracker already know that you're looking at something,
Speaker 1: so that it knows what you want to click on
Speaker 1: before you touch your mouth. So that could come in
Speaker 1: really handy for anyone who's designing something to have a
Speaker 1: process where you run the test and you have eye
Speaker 1: tracking technology, you can see what elements of your design
Speaker 1: are actually working or maybe are distracting to your to
Speaker 1: your potential user. Yes, that is exactly what to be
Speaker 1: pro that their entire business models is centered around that.
Speaker 1: For example, you know they're creating glasses eye tracking, a
Speaker 1: set of glasses and they send a test group through
Speaker 1: a store, a retail store, and then big retail companies
Speaker 1: such as Propering annulal unit level that would test the
Speaker 1: store layout and see what package did I look at?
Speaker 1: What package did I not see? And how do I
Speaker 1: reconfigure the store layout. It can also be done on
Speaker 1: websites or pack research, etcetera. But US excellent and it
Speaker 1: can also be used not just to see where our
Speaker 1: attention is, but it can go a step further and
Speaker 1: become a direct interface with some other technology. For example,
Speaker 1: you can use it so that you are creating an action,
Speaker 1: you're removing that need to move and click correct. Yes.
Speaker 1: So the first thing, the first category is just it
Speaker 1: gives computers or it gives devices insight of what you're
Speaker 1: looking at. If you're in front of the computer, are
Speaker 1: you're focusing on the screen, or you're leaving the screen,
Speaker 1: what you saw, what you did not see. It can
Speaker 1: be presence and identity in all these things. It gives
Speaker 1: you that it gives you another knowledge about the human.
Speaker 1: And the second thing is you can use it for
Speaker 1: interaction like taking an example from VR. For example, if
Speaker 1: you imagine you pick up something in something in your
Speaker 1: hand and you want to throw it, and if you
Speaker 1: do this, you throw it aim at the a's in
Speaker 1: your room. You want to throw it. When you as
Speaker 1: a human aim at that place, you are looking at
Speaker 1: the place with your eyes and then you give the
Speaker 1: command to your body to throw the thing with your hand.
Speaker 1: But the VR headset in order to tell where you
Speaker 1: want to throw the object, you either need to point
Speaker 1: at the area you want to throw it it with
Speaker 1: your forehead or you need to point at the area
Speaker 1: you want to throw at it with your hand or
Speaker 1: do a gesture, and that's very very hard to get
Speaker 1: exact basically, So this is kind of replicating the human
Speaker 1: behavior when you pick up pick up something and you
Speaker 1: want to throw at something, you look at the point
Speaker 1: you want to throw at and that's how you do
Speaker 1: your aim. And headsets with eye tracking would suddenly understand
Speaker 1: your intention. They would understand what you are trying to
Speaker 1: hit because you're looking at that point. So those devices
Speaker 1: are much smarter, and we give developers the freedom to
Speaker 1: use that and it removes that barrier from a user
Speaker 1: perspect active that any time you have any kind of
Speaker 1: interface with a machine, the more levels of of abstraction
Speaker 1: that there are between you and what you're trying to do,
Speaker 1: the more there's a learning curve, right that you have
Speaker 1: to actually train yourself how to use that device, so
Speaker 1: that the device does what you wanted to do, whereas
Speaker 1: this is kind of stripping that away quite a bit.
Speaker 1: We're trying to make it. We're trying to make it natural.
Speaker 1: That is exactly how you behave in the real world.
Speaker 1: You want to throw an object at something, you pick
Speaker 1: it up, you look, that's how you aim and then
Speaker 1: you throw, while in VR you would have to do
Speaker 1: something else. So so we're trying to make it as natural. Yeah,
Speaker 1: So what we think when you think about what devices
Speaker 1: are as of today, devices have and I'm talking about
Speaker 1: devices a community or a or smartphones or computers. But
Speaker 1: they they they know if you push the buttons or
Speaker 1: push them right. They listened to you because they have
Speaker 1: cortona or they have different types of voice input and microphones.
Speaker 1: They know where you are because they may have a GPS.
Speaker 1: They know if you turn them around because they have
Speaker 1: a gyro, etcetera. They know all these things, but they
Speaker 1: have no concept, no idea of of the human in
Speaker 1: front of them. They don't know if there's there's anyone
Speaker 1: in front of them. So we typically say that devices
Speaker 1: as of today are blind. They can listen, they can feel,
Speaker 1: they know where they are, they know in what direction
Speaker 1: they are, but they are blind. They have no concept
Speaker 1: of who is in front of them, and what is
Speaker 1: that person doing, and and where is that person looking
Speaker 1: what is that person's intention? We think that's in the
Speaker 1: in the world of AI, you know, trying to create
Speaker 1: machines that understands use humans. We think that's kind of
Speaker 1: actually crazy. Um, it is I would I would probably
Speaker 1: argue pretty strongly for that. It is it really or
Speaker 1: more important to know what is your what is my
Speaker 1: user's intention? What are they interested in at every given
Speaker 1: point in time, and knowing if they are in front
Speaker 1: of the computer, as knowing where they are or in
Speaker 1: what direction the laptop or smartphone is turned interesting. So
Speaker 1: with this, this philosophy, this and this general approach, this
Speaker 1: idea of teaching machines how we humans pay attention, you know,
Speaker 1: looking for the signs that show that we are in
Speaker 1: fact focusing upon any given aspect. What is the actual
Speaker 1: technological component that allows these machines to do that. I
Speaker 1: imagine that we're talking about a hardware side with a
Speaker 1: camera and then obviously a software side that's processing that
Speaker 1: information and making meaning out of it. Yeah, I mean
Speaker 1: the core components of a of an eye tracking system.
Speaker 1: It's it's a it's a camera in near infrared camera
Speaker 1: and it's a set of illuminators that can that protected
Speaker 1: light pattern or your eyes, and then this camera takes
Speaker 1: the pictures with this light pattern. Then you send these
Speaker 1: pictures the picture stream onto sometimes formal processing units where
Speaker 1: there is a set of algorithms that can calculate um
Speaker 1: either where you're looking, or you can do face idea
Speaker 1: on the same camera, so who you are, or it
Speaker 1: can calculate what direction your head is. It can detect
Speaker 1: your facial pictures. Are you happy, are you sad? It
Speaker 1: can take the drowsiness, are you awake or sleeping, So
Speaker 1: you can do a lot of different things with those pictures.
Speaker 1: And then the third component, it's kind of the camera,
Speaker 1: the algorithms, and then it's the use case. So what
Speaker 1: do you do with the knowledge of that you're looking here?
Speaker 1: That's kind of built into the use case, and what
Speaker 1: do you do with the knowledge that you know the
Speaker 1: identity of this person? So it might be something in
Speaker 1: the case of authentication where you're looking at a person's idea,
Speaker 1: their their facial structure. Similar to what we saw with
Speaker 1: the the reveal of the new iPhone handsets that are
Speaker 1: coming out, they're using a face i D technology for
Speaker 1: that sort of thing authentication, but it could it could
Speaker 1: also be control systems, whether it's in a game or
Speaker 1: as you had mentioned earlier, for people who have mobility
Speaker 1: disabilities that they aren't able to move, they're able to
Speaker 1: perhaps use their eyes to focus on specific icons and
Speaker 1: thus communicate using eye tracking software to indicate what they're
Speaker 1: intent is. Yeah, so you see big trend in consumer
Speaker 1: actronics right now. If I mean the iPhone is is
Speaker 1: just one of them. They're entering because they want to
Speaker 1: make their devices smarter and understand the users better. They
Speaker 1: enter special use of facing cameras, nearing for red cameras
Speaker 1: which can understand their users more. And that is exactly
Speaker 1: what iPhone did, I mean iPhones quote in one of
Speaker 1: the newspapers or articles I read was what can be
Speaker 1: more natural than touch? Well just a look. So iPhone
Speaker 1: Is or Apple is recognizing that, hey, yeah, we're using
Speaker 1: the touch interface today. But if I really won't understand
Speaker 1: my user, I need to understand what the user is,
Speaker 1: what the user is doing in front of advice. That's
Speaker 1: why they implemented face i D sure in the phone um,
Speaker 1: And that's why I mean Huawei. We did the phone
Speaker 1: with Huawei Run about a year ago which had a
Speaker 1: similar type of similar type of camera which could do
Speaker 1: various features as well and understand you as a user.
Speaker 1: Are you in front of it? Then the screen didn't live, etcetera, etcetera.
Speaker 1: It's the same thing that is happening with eye tracking.
Speaker 1: It is the same thing which is happening in the
Speaker 1: automotive industry where you have used of facing cameras to
Speaker 1: detected drowsiness. Except it's the same thing that is happening
Speaker 1: in in v R where a lot of people are
Speaker 1: are looking into eye tracking cameras or user facing cameras
Speaker 1: in in in, in the in the in the headset.
Speaker 1: So it's a it's a general trend basically. And I
Speaker 1: had a chance to experience this and many years ago.
Speaker 1: In fact, I think it was a booth at c
Speaker 1: E S and I think it was a Toby booth
Speaker 1: if I'm not mistaken, where I sat down and got
Speaker 1: a general demo of the eye tracking technology. At the time,
Speaker 1: there was an application. It was a game essentially sort
Speaker 1: of like the old game of Asteroids, where you would
Speaker 1: focus on items on the screen, and wherever your focus was,
Speaker 1: that's where the quote unquote firing of the the asteroid
Speaker 1: destroying laser or whatever it was would focus. And it
Speaker 1: was a great little demo showing the promise of it.
Speaker 1: I imagine that as all technologies do, this has evolved
Speaker 1: to become much more accurate and precise over time. Yes,
Speaker 1: so that's obviously and one of our core challenges and
Speaker 1: tusks is to improve the accuracy and precision. But it's
Speaker 1: also to make it work consistent live for the entire
Speaker 1: population in all different conditions. That is what it is hard.
Speaker 1: With an eye track there, making an eye track of
Speaker 1: the work for one person is prettycy, but making it
Speaker 1: one an eye tracker the works the same way for
Speaker 1: everyone is pretty hard. You can compare that with voice recognition.
Speaker 1: I mean, making a voice recognition system that that recognizes
Speaker 1: your voice is pretty okay, pretty easy, But making to
Speaker 1: do with it for the entire human race in all
Speaker 1: different conditions, with all different background sounds, that's pretty hard. Um.
Speaker 1: So that's that's the challenge, and that's that's what we
Speaker 1: are ultimately they're leading the world on. So you're you're
Speaker 1: looking at the separating all the signal from the noise
Speaker 1: making sure that on any I mean, I sit there
Speaker 1: and I just think about all the different cases where
Speaker 1: I'm using my computer. I mean, even if it were
Speaker 1: just a laptop implementation, keeping all the others aside just
Speaker 1: the laptop alone. There are times where I'm at my desk,
Speaker 1: there are times where I'm sitting at a couch or
Speaker 1: in another environment. They're different lighting conditions, they're different angles. Um.
Speaker 1: I imagine that that for most of these there's probably
Speaker 1: some sort of UH training or orientation UH segment where
Speaker 1: it allows you to to at least set that baseline
Speaker 1: for eye tracking, for to make sure that you know
Speaker 1: you're getting a good response early on. But yes, as
Speaker 1: you point out, if you're rolling out of technology that's
Speaker 1: going to go to a wide consumer market, then that
Speaker 1: challenges is non trivial. You have something that needs to
Speaker 1: work out of the box for UH an enormous really
Speaker 1: an endless variety of people in and an almost equally
Speaker 1: endless variety of situations. So um, these days you've mentioned
Speaker 1: some of the tools. One of the things I was
Speaker 1: really interested in was this user interface with computers specifically.
Speaker 1: But I mean, obviously all the differ friend. Uses of
Speaker 1: eye tracking are are fascinating to me. Um. One of
Speaker 1: the things I was really kind of interested in is
Speaker 1: the implementation in the sense of how it works for
Speaker 1: uh in the pro gaming industry. We've been looking at
Speaker 1: that recently, and UM, I think it's really cool to
Speaker 1: have a new tool to analyze the way people who
Speaker 1: can play at a professional level. How you know where
Speaker 1: their attention is and is it different from people who
Speaker 1: play at a regular gaming level, like someone like me.
Speaker 1: I consider myself a bullet sponge when it comes to games.
Speaker 1: I'm I'm the one that people practice on. Yeah, no,
Speaker 1: I'm great at doing that. I'm I'm if you have
Speaker 1: if you have a healer on your team who needs practice,
Speaker 1: I'm the guy who's gonna be giving that guy lots
Speaker 1: of work. But uh, I'm sarious. How have you seen
Speaker 1: this rolled out and implemented? Have Have there been any
Speaker 1: things in that implementation that have surprised you or any
Speaker 1: interesting stories through that process? I mean this, this entire
Speaker 1: story is based on what we do in in in
Speaker 1: what we have done many many years in various business units,
Speaker 1: and it's based on the notion that what you look
Speaker 1: at is a good approximation of what you think because
Speaker 1: that is your intention. So the entire concept is pretty
Speaker 1: straightforward in that. And when you when you think about that,
Speaker 1: you can if I mean a pro gamer, where that
Speaker 1: person spends his or her attention is extremely precious. I mean,
Speaker 1: they have so many different impressions coming at them at
Speaker 1: the same time, and they need to select and need
Speaker 1: to be very good at making sure they perceived the
Speaker 1: right things, making sure they spend their time on the
Speaker 1: right things, otherwise they they will get killed. Um so,
Speaker 1: and the concept is pretty much going to link to that. So,
Speaker 1: for example, a pro gamer may know that they need
Speaker 1: to regularly check check the minima. You know, how many
Speaker 1: how many times a minute does a pro game or
Speaker 1: check the minimap. You're going to see them continuously going
Speaker 1: down there and checking what's happening, while a less you know,
Speaker 1: a less good gamer may focus too much on the game.
Speaker 1: You can you can you can check that out and
Speaker 1: see how many times are you're actually checking minima or
Speaker 1: you can see pro gamers are also before he's attacking,
Speaker 1: checking the enemy items. You know, you know, what type
Speaker 1: of skills or what type of weapons or what type
Speaker 1: of skills does does the opponent have right now, you
Speaker 1: can see that a less less good game it may
Speaker 1: not do that. Yeah, you can also go into different
Speaker 1: players player styles. You know, when you are a certain
Speaker 1: player style, you may have a different games pattern than
Speaker 1: you are another player style because you need you need
Speaker 1: to do that differently. Or you can see things like
Speaker 1: keyboard combination. I mean, you know, as as a really
Speaker 1: good game and you know you need to learn all
Speaker 1: your keyboard combinations by heart. You need to do that, um,
Speaker 1: and that's gonna be and by doing that, you can
Speaker 1: spend all your time focusing on the focusing on the screen. Well,
Speaker 1: less good gamers they will not be they will not
Speaker 1: know all the keyboard combinations. They will be focusing off
Speaker 1: screen for a certain percentage of the time. You can
Speaker 1: notice that. So you can kind of say, hey, you
Speaker 1: need to practice your keyboard combinations because your off screen
Speaker 1: thirty example of time or trying to perstand the temperation
Speaker 1: on whatever it is you know. Um, so it's it's
Speaker 1: it's things like that that you can notice that can
Speaker 1: make you become a better gamer. If you study this
Speaker 1: is my favorite gamer, this is how they behave I
Speaker 1: see how they are playing and what they're intent is,
Speaker 1: then you can learn from that. You may not have
Speaker 1: the same players toty hole, but you can learn from
Speaker 1: that and practice on certain certain events. Yeah, it might
Speaker 1: even just teach you what parts of the game you
Speaker 1: don't need to worry about because those are purely environmental
Speaker 1: and have no no direct impact on whether or not
Speaker 1: there's an opponent, and therefore you don't waste time looking
Speaker 1: at something that is not a threat exactly. And you
Speaker 1: can see that just in the in the game where
Speaker 1: somebody is playing without tracking, they can turn on what
Speaker 1: we call a gaze overlay, and then where they're looking
Speaker 1: you will get the kind of transparent blob. If the
Speaker 1: streamer wants you to see that, so they can turn
Speaker 1: it on, and then you, as as a viewer, get
Speaker 1: the much richer experience because you can suddenly understand that,
Speaker 1: all right, I know not only where the party, where
Speaker 1: the where the pro is clicking, but I also know
Speaker 1: where he's spending his attention, so I kind of can
Speaker 1: anticipate his next move because I can see a little
Speaker 1: bit how he thinks interested that, and that will make
Speaker 1: you anticipate his next move and make you understand his
Speaker 1: gameplay better and therefore you can always game and therefore
Speaker 1: learn more. So just by looking how they scan off
Speaker 1: the environment, you can kind of follow how the pro is,
Speaker 1: how the pro is perceiving and playing the game, which
Speaker 1: is incredibly useful for us as a as a as
Speaker 1: a as a game that want improved. Yeah, this kind
Speaker 1: of as a parallel. This reminds me a lot actually
Speaker 1: of poker tournaments that are televised, where you get a
Speaker 1: chance to see what the cards are of any particular player,
Speaker 1: which then lets you understand more about the decisions the
Speaker 1: players are making as the game unfolds. This is kind
Speaker 1: of the equivalent to that you're not just watching the
Speaker 1: game happen. You're actually seeing how the person playing it is.
Speaker 1: You know what what they are perceiving and what is
Speaker 1: important to them, and that's why they make the decisions
Speaker 1: they make. So now you're not just seeing their decisions,
Speaker 1: but you see why those decisions are happening exactly. I mean,
Speaker 1: you you can take it to the to the you know,
Speaker 1: real world poker world. If you're sitting around the you know,
Speaker 1: really fiscal table, then you really good poker players well
Speaker 1: will of course be looking at their cards, but they
Speaker 1: would probably spend a very large portion of the time
Speaker 1: watching their opponents. If you had eye tracking on them,
Speaker 1: you could actually see what are they focusing on, what
Speaker 1: parts of the player are they focusing on, and when
Speaker 1: you know and how do they know? So you can
Speaker 1: see what is it that they're focusing on, and what
Speaker 1: are the interesting areas. As a non pro gamer, you
Speaker 1: would probably learn immensely by knowing, all right, I should
Speaker 1: not focus so much on my own cards, I should
Speaker 1: focus on new opponents x percent at the time. And
Speaker 1: then this is what the pros are actually looking at.
Speaker 1: He look, he's lick, he's looking at if player he
Speaker 1: is looking at player B and seeing if plays these
Speaker 1: blinking and if he's blinking, he's kind of folding or doubling. Yeah,
Speaker 1: so because because because that is the queue. Yeah, that's
Speaker 1: and it's you know, that's incredible. It's it's blowing my
Speaker 1: mind to even think about that possibility. Because we know
Speaker 1: that people who or form at that at that professional level,
Speaker 1: at that level that is above like sometimes a huge
Speaker 1: leap above what we would consider a good player. There
Speaker 1: are these these components and people always talk about tells
Speaker 1: and they talk about the psychology of the game. But
Speaker 1: that's stuff that is really difficult to learn unless you
Speaker 1: are pouring hundreds of hours of experience in to be
Speaker 1: able to see something like this, where you can get
Speaker 1: that direct feedback of oh, well, I understand conceptually what
Speaker 1: they're doing. Now I can actually see how they are
Speaker 1: doing it. Even though I would argue it would still
Speaker 1: take you many hours to to get a level of
Speaker 1: proficiency there, you would at least have a place to start. Yeah, exactly.
Speaker 1: But you can take this also too, I mean, can
Speaker 1: take this to the real sports world. We have race
Speaker 1: Formula one race car drivers were NASCAR drivers who are
Speaker 1: using eye tracking in their helmets in order to see
Speaker 1: rick chord where they are in practice now, where they
Speaker 1: are looking and when they take the turns, because one
Speaker 1: of the most important things going to take a turn
Speaker 1: is to have the right kind of direction of your right,
Speaker 1: right kind of focus points. And they're kind of learning
Speaker 1: from the really good drivers to the less good drivers.
Speaker 1: Where is the focused point in the term. So because
Speaker 1: they can see right the good drivers are behaving like this,
Speaker 1: this is their focused point. You know you should change.
Speaker 1: You're focusing on their own directions. So it's kind of
Speaker 1: it's it's in the physical world as well. Two. I
Speaker 1: mean you can see the poker examples very easy, and
Speaker 1: then you can obviously see the the the the the
Speaker 1: the the computer game example as well. So this is
Speaker 1: a two way street, right, We're talking about in one
Speaker 1: side of things having a new approach, relatively new, I mean,
Speaker 1: eye tracking has been around for a while, but to
Speaker 1: see the technology mature to a point where it's rolling
Speaker 1: out more more consistently. In consumer market stuff, we see
Speaker 1: a new user interface approach. But on the other side,
Speaker 1: we're seeing a way of learning more about ourselves. And
Speaker 1: so it's really cool. It's technology that you can take
Speaker 1: advantage of as a developer to create a new way
Speaker 1: to interact with a product or whether it's hardware or software.
Speaker 1: And then on the flip side, from a psychology standpoint,
Speaker 1: you can learn more about human behavior. You can learn
Speaker 1: more about human behavior and extremes such as the extreme
Speaker 1: level of performance with professional gamers and that kind of thing. Um. So,
Speaker 1: to me, this is an amazing technology, not just for
Speaker 1: the potential of what I can do with it, but
Speaker 1: what ultimately we can learn about ourselves and thus design
Speaker 1: even better stuff another generation or two generations down the line, Yeah,
Speaker 1: I'm out in just a computer developer, if they could see,
Speaker 1: if they take a sample of a thousand gamers and
Speaker 1: users of the game, and they would know on what objects,
Speaker 1: what what are these guys are in girls seeing in
Speaker 1: the user interface? And what are they not seeing? Alright,
Speaker 1: they're not noticing I mean, they're not clicking on this thing.
Speaker 1: They see it, but they turn away because because they're
Speaker 1: not interested in it, or they're just missing this thing.
Speaker 1: You know, how what what cues can you give it
Speaker 1: to you? So if you know what they have, what
Speaker 1: they have seen, you don't need to have a second
Speaker 1: reminder of some of something that somebody has seen, seen,
Speaker 1: seen already. So you can improve the user interface immensely
Speaker 1: obviously by the stuff of right. And of course today
Speaker 1: now that we're in a world where people have persistent
Speaker 1: Internet connections, it allows you to roll out uh tweaks.
Speaker 1: You can roll out versions and patches on a very
Speaker 1: frequent basis. This allows for incredible flexibility if you are
Speaker 1: trying to create something that is software based and you
Speaker 1: are getting real time feedback due to the eye tracking
Speaker 1: data about what is and isn't working, or what is
Speaker 1: or isn't necessarily considered were the a focus and either
Speaker 1: redesign it so that something that you think is important
Speaker 1: can have a better sense of prominence in the eyes
Speaker 1: of the people using it, or you just eliminate it.
Speaker 1: If no one's looking there and it's taking up landscape,
Speaker 1: you might just get rid of that entirely, if no
Speaker 1: one really is using it. It's to me, it's like
Speaker 1: another leap forward where we saw advertising go once the
Speaker 1: Internet became a real entity. Because of course, in the
Speaker 1: old days, advertising was limited to print and radio and
Speaker 1: television uh campaigns, and you had a sense of how
Speaker 1: well your ad campaign was going based upon sales, but
Speaker 1: it was really hard to nail down whether or not
Speaker 1: that was actual causation, right, because maybe it was because
Speaker 1: the ad campaign, or maybe there was some incredible sale
Speaker 1: that really help boost sales. But then the Internet comes
Speaker 1: along and that gives a much more direct feedback loop
Speaker 1: of what ads are are encouraging people to click through
Speaker 1: and therefore experience it on a deeper level. This is
Speaker 1: like that, except it can apply to everything, not just advertising,
Speaker 1: but everything. So that's that's what I mean when when
Speaker 1: I say eye tracking gives understanding of your intent and
Speaker 1: how important is attempt to any any any computing device.
Speaker 1: Intent is is probably the most important courage in that
Speaker 1: you have. You know, you're you're you're kind of intent
Speaker 1: and you're interesting. Your attention, I mean that is the
Speaker 1: most the attention of the of humans is the most
Speaker 1: pressured pressure pressures, pressure resource you have. Right, but then
Speaker 1: it jumps down that, right, you can never share eye
Speaker 1: tracking data the user doesn't want it show. I mean,
Speaker 1: it's it's the user that controls it, so so there's
Speaker 1: no kind of possibilities to spy. It's like you as
Speaker 1: a user, you control if you want to give this
Speaker 1: to a corporation or if you don't basically like anything else.
Speaker 1: That's super super important. Obviously, absolutely yes you wouldn't if
Speaker 1: you're volunteering it, that's one thing. But if you just
Speaker 1: found out, oh gosh, Facebook implemented this technology and now
Speaker 1: they know everywhere I look, and now my feed is
Speaker 1: being filled with puppies, and I like puppies, but I
Speaker 1: really would like other things besides puppies. Uh. I mean,
Speaker 1: obviously we see that sort of analysis in all areas
Speaker 1: of technology that are going on, just based upon our
Speaker 1: actual physical behaviors. Are choices made, like whether we're following
Speaker 1: one link versus another link. But I think a lot
Speaker 1: of people would really uh, if it weren't something that
Speaker 1: they were volunteering, they would really have a negative reaction
Speaker 1: to that, to think, oh, well, now it's gone beyond
Speaker 1: what I'm choosing to do. Now it's going to what
Speaker 1: I'm I'm actually looking at. And uh, I think a
Speaker 1: lot of people would find that creepy if it weren't
Speaker 1: a volunteer, voluntary thing. So I agree with you entirely
Speaker 1: that that that's an important step for this to be
Speaker 1: a technology that people embrace as opposed to reject. Yeah,
Speaker 1: and that's going we put hard viruses with the guardians
Speaker 1: of the end users privacy or when you can only
Speaker 1: read it if you're if you're loving right. So what
Speaker 1: I want to ask you what your personal favorite implementation
Speaker 1: of eye tracking technology has been up to this point.
Speaker 1: It's it's a very good question. It's also very hard
Speaker 1: question because as we have discussed, his knowing the intent
Speaker 1: of humans is so powerful, So you can you can
Speaker 1: drew up so many different example and implementations. But if I,
Speaker 1: if I, if I take a few, um we have
Speaker 1: in our business unit Tobo Dona woks. We have so
Speaker 1: many users. So when their mothers and fathers who come
Speaker 1: to us and say, hey, my my son or my
Speaker 1: daughter has for the first time ever communicated to me, mom,
Speaker 1: I love you, or has for the first time drawn
Speaker 1: a drawing on on on anything, but this time on
Speaker 1: the computer with with his or her eyes, I mean
Speaker 1: that that goes beyond anything when you get that type
Speaker 1: of really really kind of strong value, because because a
Speaker 1: person who has previously not been able to communicate can
Speaker 1: now communicate, I mean that that's kind of that's outside anything. Yeah,
Speaker 1: that's transformative. I mean that's that's the sort of story
Speaker 1: that is incredibly inspiring, and it does show that this
Speaker 1: technology has an incredible power, and it can not only
Speaker 1: does have incredible power, it has incredible power to empower
Speaker 1: people who previously had very little chance of having that
Speaker 1: sort of human connection. And until you hear those stories,
Speaker 1: you don't even realize how incredibly valuable that can be
Speaker 1: and how much those of us who haven't had to
Speaker 1: experience that take it for granted. But then you start
Speaker 1: thinking about it and you realize this sort of connection
Speaker 1: that is possible due to technology like this and you really,
Speaker 1: I mean, it really does open up your eyes on
Speaker 1: didn't mean to go with that particular metaphor, but that
Speaker 1: it really it really does does hammer home how how
Speaker 1: incredibly impactful this sort of technology can be. The other
Speaker 1: the other if I take one more example, more from
Speaker 1: the consumer world. You know, I don't think it's hard
Speaker 1: to tromp that type of but take it from the
Speaker 1: consumer world. If I think about a VR headset or
Speaker 1: or an a R or a R air headset, it's
Speaker 1: what what is happening is we are actually seeing that
Speaker 1: we can take a three step interaction and make it
Speaker 1: into two step interaction and take away one entire step
Speaker 1: into every single interaction. So interview had set us of
Speaker 1: today the method to interact with something is one, you
Speaker 1: look at it too, you turn your head or you
Speaker 1: point your hand controller to it. Three, you push a
Speaker 1: button to pick it up or you know, whatever you
Speaker 1: want to do with it. So it's kind of one
Speaker 1: look to turn your head or move your hand control
Speaker 1: to the point, three press a button. You know. Their
Speaker 1: variations could be my voice as well, but there so
Speaker 1: that's a three step process. With eye tracking, you reduced
Speaker 1: that to two step process, which is basically one you
Speaker 1: look and an object to your press. That's fantastic, and
Speaker 1: it creates a deeper sense of immersion because you're no
Speaker 1: longer again you're removing another one of those layers that
Speaker 1: the person using it had to go through previously. Every
Speaker 1: time one of those layers gets removed, then you have
Speaker 1: this this deeper immersing of the sense that creates a
Speaker 1: more convincing experience overall. Yeah, and when you think about that,
Speaker 1: what when you think about what the entire touch revolution,
Speaker 1: what that really is? You know, you know I when
Speaker 1: you've given a touch phone to you, you know, your
Speaker 1: first experience with a touch phone with and and and
Speaker 1: the you know the and the what that revolution is
Speaker 1: is like one you look to your touch. That is
Speaker 1: what you do on the touch phone. It's a two
Speaker 1: step in the direction one too, just like it is
Speaker 1: in the order with eye tracking, one to press about
Speaker 1: them while it's coming from the PC world, where the
Speaker 1: interaction is one look to drag with your mouth or
Speaker 1: your touch pad three click. So the revolution you're going
Speaker 1: through with eye tracking is is exactly the same in
Speaker 1: terms of taking one step away and just having it
Speaker 1: one look to action, one look to action. So it's
Speaker 1: I would argue, it's just as powerful as that touch revolution.
Speaker 1: And when you think, when you think about it further,
Speaker 1: why is this so much more intuitive? It is because
Speaker 1: that is how we have learned to interact, interact even
Speaker 1: since we were a kid. You know, when you, as
Speaker 1: a kid wanted to learn we can interact, to interact
Speaker 1: with something. You have something on the table in front
Speaker 1: of you, it's one look to pick up to explore.
Speaker 1: You know, when you want to turn you on your dishwasher,
Speaker 1: I mean one look at the knob to turn. When
Speaker 1: you want to shake somebody hand, look at the one
Speaker 1: look at the hand to shake the hand. You know,
Speaker 1: it's it's a two step interaction. That is what you
Speaker 1: have learned as a kid to explore, one look to
Speaker 1: pick up and and we're replicating that. And that's why
Speaker 1: it feels so intuitive. And I can tell you anybody
Speaker 1: that we demo a VR or or are application to
Speaker 1: they kind of they get it instantly, like, oh yeah,
Speaker 1: this is more intuitive because it's simple as that we're
Speaker 1: just replicating what you do in the red world, which
Speaker 1: is which is kind of actually, if you wanted to
Speaker 1: virtual reality, it's it's supposedly pretty important to replicate your
Speaker 1: actress in the real world, right, yes, I mean if
Speaker 1: it If it doesn't, then it takes you out of that.
Speaker 1: You're aware that you are in an artificial experience. And
Speaker 1: uh And obviously the goal is to reduce that awareness
Speaker 1: as much as possible so that you can really have
Speaker 1: that the experience of the developer intended when he or
Speaker 1: she started to design it. Uh And and ideally, at
Speaker 1: least for most VR experiences, that's to strip away all
Speaker 1: the rest of the awareness in augmented reality. Obviously it's
Speaker 1: meant to augment and experience in the real world, but
Speaker 1: even then, you want to you want to reduce the
Speaker 1: load on the user of having to think, oh, well,
Speaker 1: if I want to accomplish goal X, I have to
Speaker 1: go through these steps in order to actually do that.
Speaker 1: Whereas in the real world, if you were looking at
Speaker 1: an actual physical rep of what you were seeing in
Speaker 1: the argumented world, you wouldn't have to go through that
Speaker 1: because that's just not the way the real world world works.
Speaker 1: Imagine imagine, imagine a fast paced VR game or a
Speaker 1: R game, it doesn't matter. I mean, you would be
Speaker 1: running down a pitch in a in a football match,
Speaker 1: and you want to pass the ball to somebody who's
Speaker 1: running full speed in the other direction, and then you
Speaker 1: how do you do that with the current controls in
Speaker 1: the R I mean you're running full speed your yourself,
Speaker 1: and then you should either point your head to kind
Speaker 1: of to to point to the other, to the other,
Speaker 1: to your teammates running in the other direction, and do
Speaker 1: that continuously to tell the computer where you want to
Speaker 1: tell your head that way you want to pass the ball,
Speaker 1: or you should running full speed one direction. You should
Speaker 1: point your hand to tell the via heads that I
Speaker 1: want to pass the ball to this guy. That's super hard.
Speaker 1: I mean, that's just gonna It's just impossible. You can
Speaker 1: try to do it yourself in your any room if
Speaker 1: you want. But or you imagine the interaction. You're running down,
Speaker 1: you know, in one direction, you look at the guy
Speaker 1: running in the other direction. You can do that. You
Speaker 1: can follow that person extremely easily with the rice and
Speaker 1: then you just press a button then post that a guy.
Speaker 1: That's that's just so much easier, and it takes down
Speaker 1: this barrier. It takes away one step and it will
Speaker 1: enable game developers to make Vio games more immersive and
Speaker 1: also more frost pace, because otherwise you need to kind
Speaker 1: of slow down the pace in order to make people
Speaker 1: hit the right person. And not only that, I would
Speaker 1: add that it adds in an element that you see
Speaker 1: in the real world all the time, which is misdirection,
Speaker 1: purposeful misdirection. If you're playing on a team and you
Speaker 1: want to pass to a teammate, you don't want to
Speaker 1: telegraph that to the other team. So so being able
Speaker 1: to use your attention to direct the ball in the
Speaker 1: in the right direction without having to turn your head
Speaker 1: and make it obvious, oh well, he's about to pass
Speaker 1: to his teammate on his left, then it makes it
Speaker 1: more challenging for the opposing team to anticipate what your
Speaker 1: move is. Yeah, or more I mean more like personal
Speaker 1: connecting interaction. I mean you're sitting around the table, you're
Speaker 1: talking to set up people. Imagine you have two people
Speaker 1: in front of you now, and then you start talking
Speaker 1: to one of them. You would be looking at that person, um,
Speaker 1: but you would be looking at that person with your eyes.
Speaker 1: You would not turn your forehead to that person. So
Speaker 1: the v R A R head said would not know
Speaker 1: that you're you weren't talking to person A. It may
Speaker 1: you may have your head the point still the point
Speaker 1: that that person be. So then the via head said,
Speaker 1: would it would the wrong person would not or blink
Speaker 1: or or or or smile at you when you when
Speaker 1: you smile back at them, because they you don't know
Speaker 1: what you're what, what you're interesting. You only know what
Speaker 1: you're interested in, what you're interested in if you know
Speaker 1: where you're looking, because your forehead is just an approximation
Speaker 1: of your interest, and it's always it's going to be
Speaker 1: wrong in in certain number of cases. Oh sure. Yeah.
Speaker 1: So if you were in a game that wire's interaction
Speaker 1: with uh PC controlled characters, and you're trying desperately to
Speaker 1: uh to align yourself with one character, and there's a
Speaker 1: secondary character there, and you accidentally swear fealty to the
Speaker 1: bad guy, Suddenly the whole game has changed, yeah, or
Speaker 1: just feels less immersive because the people you look at
Speaker 1: their stone face, they don't want to spot at them
Speaker 1: because they don't understand they're looking at them because you didn't.
Speaker 1: You didn't do you didn't turn your forehead to them
Speaker 1: to indicate that you actually want to speak to them,
Speaker 1: which is I'm not sure you don't do that really
Speaker 1: well and I could see this also just being useful
Speaker 1: in a virtual meeting space where you're just having a
Speaker 1: like just a not a game element at all, just
Speaker 1: a literally a meeting where you have multiple people logging
Speaker 1: in through various devices. Being able to see where anyone's
Speaker 1: gaze is is incredibly useful because if you're addressing questions
Speaker 1: to somebody, you don't have to It's it's kind of
Speaker 1: like inter acting with a personal digital assistant that's voice
Speaker 1: activated today where uh, you know you. You use a
Speaker 1: typically a name that indicates this is the alert to
Speaker 1: listen in and then to respond. But it's not the
Speaker 1: way we humans tend to talk to one another. If
Speaker 1: I were having a conversation with you, and I started
Speaker 1: every sense with Oscar, what do you think about after
Speaker 1: about after about five or six sentences, it feels weird.
Speaker 1: So you've got it. You've got it. Can I hire you? No?
Speaker 1: But that's that's it. You can I take one more
Speaker 1: use case, just absolutely please do or another I mean
Speaker 1: just to take it in a different direction or tangent.
Speaker 1: It's it also can also affect the actual core rendering
Speaker 1: of the graphics. So I mean today in VR or
Speaker 1: in PC, I mean, we're all creating to create beautiful
Speaker 1: games with with as high resolution as possible, but that
Speaker 1: takes a high toll on on the on the GPU
Speaker 1: mhm um. But actually the human eyes only perceiving what
Speaker 1: you look at in high resolution where you look at it,
Speaker 1: the rest is kind of blurring. So that means you
Speaker 1: can do something called phobid rendering, that is that you
Speaker 1: only render in high resolution in the area you look
Speaker 1: and in low resolution in the prefery, and then you
Speaker 1: look at a different point, and then it switches very
Speaker 1: fast to render in high resolution where you look at
Speaker 1: that point and in lower resolution and periphery. As a user,
Speaker 1: you cannot notice because this is all happening too fast.
Speaker 1: You you're not noticing that the entire world is rendered
Speaker 1: in lower resolution in the prefery. But you save thirty
Speaker 1: or two on the GP capacity excellent. So then that
Speaker 1: also means that you're not overheating your machine. It means
Speaker 1: that devices that have say the latest and greatest in
Speaker 1: in graphics hardware, that's going to remain relevant longer that
Speaker 1: it would be like the life cycle of these things
Speaker 1: is notoriously fairly short. If you want to be on
Speaker 1: the cutting edge of capability. But if you're using a
Speaker 1: strategy such as this that reduces that load, then you've
Speaker 1: extended the life, the useful lifespan of those uh, those
Speaker 1: those components by quite a bit because you don't you're
Speaker 1: not maxing out as quickly as you would in the
Speaker 1: in the real world. I mean I that was one
Speaker 1: of the reasons why back in the nineties gaming computers
Speaker 1: got to the first they became a thing, and then
Speaker 1: they became a thing that I just could not keep
Speaker 1: up with. And it was very frustrating for me to
Speaker 1: think that every six months my machine was out of
Speaker 1: date enough where if I went out and purchased a
Speaker 1: new game six months after I bought a machine, or
Speaker 1: I bought a graphics card, I could not run it
Speaker 1: at the settings I wanted to because it would be
Speaker 1: too strong, to too big a strain on my components.
Speaker 1: I would have to actually upgrade my hardware. So if
Speaker 1: you're able to solve that through software where you're you're
Speaker 1: able to reduce that load, to me, that's phenomenal because
Speaker 1: it takes me out of that massochistic relationship I had have.
Speaker 1: You can be also be a little bit less of
Speaker 1: a bullet sponsor. Right, Oh, yes, that would also, that
Speaker 1: would be lovely. I mean, let's not dream too big here, Oscar.
Speaker 1: Let's let's keep our expectations realistic. No, but you can.
Speaker 1: You can use it obviously to keep your hardware longer
Speaker 1: or to reduce your cost of your hardwork. Or you
Speaker 1: can use it to get a better immersion by being
Speaker 1: able to get your computer to handle high resolution graphics.
Speaker 1: Or you can get your computer to um to run
Speaker 1: at the higher refresh rate or higher FPS. So you
Speaker 1: can use it, you know, however you want either to
Speaker 1: save cost or prolong your prolong your lifetime, or or
Speaker 1: two to to to run the game in a in
Speaker 1: in higher solution and get more Marcy, that's fantastic, Oscar.
Speaker 1: Thank you so much for joining me on this podcast.
Speaker 1: This has been a fascinating conversation. I very much appreciate
Speaker 1: your time. Alright, when we come back, we'll talk a
Speaker 1: bit more about eye tracking technology and where it's going.
Speaker 1: But first let's take another quick break to thank our sponsor.
Speaker 1: As Mr Werner pointed out, the data we generate through
Speaker 1: our attention is valuable, and it also poses a clear
Speaker 1: threat to our privacy if it is handled poorly. Now,
Speaker 1: I appreciate that Toby's policy is to only use data
Speaker 1: with the permission of the user, because you can easily
Speaker 1: imagine scenarios which could be compromising if you did not
Speaker 1: realize your eye movements were being tracked. So, for example,
Speaker 1: let's say they're going in for a job interview for
Speaker 1: a big tech company, and you're qualified, and you are
Speaker 1: knowledgeable in your field. You are a hard worker and
Speaker 1: you really really want this job, but you're also a
Speaker 1: bit nervous about going in and interviewing. That's to be expected,
Speaker 1: and because you're nervous, you know, you're looking around a lot.
Speaker 1: You don't really want to hold anyone's eye contact for
Speaker 1: too long because you don't want to come across as
Speaker 1: too intense or anything. So you know, you're you're just
Speaker 1: trying to feel your way through this experience as best
Speaker 1: you can. And unbeknownst to you, but beknownst to us,
Speaker 1: the people meaning with you are also using a system
Speaker 1: with eye tracking technology to observe you throughout the interview process,
Speaker 1: and the system is analyzing you as you respond to
Speaker 1: questions and converse with others. It's watching your eye movements
Speaker 1: and tracking that against various behavior patterns and drawing some conclusions,
Speaker 1: and maybe some software is generating a report that assigns
Speaker 1: metrics to fuzzy categories just assertiveness or honesty, and maybe
Speaker 1: even though you really are the best candidate for the job,
Speaker 1: the system tells the hiring manager that you're nothing more
Speaker 1: than a shifty peat and they shouldn't hire you. Now
Speaker 1: that kind of sounds silly the way I'm putting it,
Speaker 1: but it's certainly something that could potentially happen, and the
Speaker 1: technology already exists. Really, it just means that you have
Speaker 1: to create the software, which largely would mean making sure
Speaker 1: that your software is at least on the surface, dependent
Speaker 1: upon reasonable psychology. Right. You can't just say I think
Speaker 1: that if you look up, you are lying. That's one
Speaker 1: of those little bits of folk wisdom that actually isn't true.
Speaker 1: So you would want to make sure that your software
Speaker 1: is based off of science, or at least appears to
Speaker 1: be based off of science. I mean, if you're not
Speaker 1: being very ethical, you could sell your software package no
Speaker 1: matter what. But it's easy to imagine that sort of
Speaker 1: scenario actually happening. That a company be that assertive or
Speaker 1: aggressive and its hiring strategies to incorporate that level of
Speaker 1: screening when they're looking at potential employees. But it's not
Speaker 1: really fair and it certainly seems invasive. Now that's the hypothetical.
Speaker 1: It's not something that I'm actually referring to that's happened
Speaker 1: in the real world to my knowledge, but it's the
Speaker 1: sort of stuff we don't want to see. It places
Speaker 1: people at a disadvantage, and it could also be used
Speaker 1: to create opportunities for abuse in many ways, which again
Speaker 1: is why that policy of only collecting information with the
Speaker 1: express permission of the user is important. You wouldn't want
Speaker 1: to come back at somebody and say, hey, I noticed
Speaker 1: that you were spending an awful lot of time looking
Speaker 1: at this one person in the room. Uh. It makes
Speaker 1: us uncomfortable because it appears that you have some sort
Speaker 1: of attraction to or maybe animosity towards this person. All
Speaker 1: of that seems like it could go south in a hurry,
Speaker 1: and it be reflective of something that is not actually real.
Speaker 1: Because while our eyes are an indicator of where our
Speaker 1: attention happens to be, it's not always the case that
Speaker 1: what our eyes are pointing at is what we're actually
Speaker 1: focusing on. It's most of the time that's the case,
Speaker 1: especially when we're being very active with our attention, but
Speaker 1: it's not always the case. So this is a complicated issue.
Speaker 1: I think eye tracking tech has far too much potential
Speaker 1: to do great things for us to be too wary
Speaker 1: of it. I don't think we need to back off
Speaker 1: of it. I don't think we need to abandon it.
Speaker 1: I don't think we need to label it as irresponsible
Speaker 1: or dangerous technology. We just have to make sure we
Speaker 1: hold organizations, companies, researchers accountable for responsible implementations of the tech.
Speaker 1: But these applications really could transform our world, particularly for
Speaker 1: people who have difficulty interacting with others or with their
Speaker 1: environment if they don't have that kind of technology at
Speaker 1: their disposal. So, like Mr Werner said, a technology that
Speaker 1: helps someone communicate after it appeared that all such ability
Speaker 1: had been lost is an incredibly powerful story, and it's
Speaker 1: hard to think of something more impactful than that. On
Speaker 1: the research side, scientists are using eye tracking to learn
Speaker 1: more about cognitive development. There's a researcher named Alex Willard
Speaker 1: who is working on her PhD and Maternal infant directed
Speaker 1: speech at the University of Newcastle and Hunter Medical Research
Speaker 1: Institute in New South Wales, and she's using eye tracking
Speaker 1: technology to study how babies develop cognitive skills. She's learning
Speaker 1: how babies recognize and solve problems. And I didn't even
Speaker 1: know babies could do that. I haven't seen Baby Boss though,
Speaker 1: so maybe it's all explained in that movie. Anyway. The
Speaker 1: study involves babies watching animations on greens and it tracks
Speaker 1: where their eyes go. So, for example, let's say you've
Speaker 1: got a cute cartoon bunny hopping around on the screen,
Speaker 1: and the baby notices the bunny and the baby's eyes
Speaker 1: are tracking where the bunny is going on the screen,
Speaker 1: and then the cartoon bunny hops behind a cartoon bush.
Speaker 1: Now at that point does the baby's focus continue along
Speaker 1: the bunny's pathway to the other side of the bush,
Speaker 1: anticipating the bunny's reappearance, And if not, why not? Now?
Speaker 1: Early results in the research project, and this project is
Speaker 1: scheduled to last for five years. It's a long term
Speaker 1: project to make sure that any findings are things that
Speaker 1: that can be backed up with replication. Some of the
Speaker 1: early research has suggested that babies who are less distractable
Speaker 1: when focusing on a subject tend to be better at
Speaker 1: problems solving later on, which makes sense intuitively, you would
Speaker 1: imagine that to be the case. But this approach could
Speaker 1: help parents detect potential challenges early on and address them
Speaker 1: and give their children the best chance of overcoming those
Speaker 1: challenges as they grow older. So you might see that
Speaker 1: certain cognitive development might be running a little bit behind
Speaker 1: average from you know, the mean, and you might say, well,
Speaker 1: what can we do to address that? Are there things
Speaker 1: we can do as parents to help encourage our child's
Speaker 1: cognitive development in these areas and thus give your kid
Speaker 1: the best chance for success. It's kind of cool, I think,
Speaker 1: and it gives a lot more information and power to parents,
Speaker 1: stuff that people didn't have at their disposal without tools
Speaker 1: like this eye tracking tech that can back up those observations. Now,
Speaker 1: another study sounds ominous upon first glance. I was reading
Speaker 1: headlines and this one jumped out at me and it
Speaker 1: made me pause. It was the National Institute of Health
Speaker 1: spending four thousand, two hundred twenty dollars on eye tracking
Speaker 1: technology to analyze the eye movements of Latino customers at
Speaker 1: grocery stores. That was essentially the headline which made me say,
Speaker 1: what is going on here that sounds minority report esque,
Speaker 1: but upon further observation further investigation, it's not nearly that scary.
Speaker 1: It's actually kind of interesting. In this case. This is
Speaker 1: a study at San Diego State University that has the
Speaker 1: goal of finding ways to fight obesity, and the system
Speaker 1: is meant to provide data about the decisions that overweight
Speaker 1: or obese people are making as they shop for groceries,
Speaker 1: and it's in an effort to get insight into what
Speaker 1: is causing them to make those decisions. Are there specific
Speaker 1: things in the grocery store environment that are triggering those decisions.
Speaker 1: Can adjustments to the environment help guide customers into making
Speaker 1: more healthy choices? And the reason the study is looking
Speaker 1: at the Latino population in particular is that, according to
Speaker 1: the researchers, as a demographic, Latinos are disproportionately affected by obesity.
Speaker 1: They also shop more frequently than other populations do, and
Speaker 1: they tend to go shopping with children more frequently than
Speaker 1: other demographics do. So the hope is that by using
Speaker 1: the information from this study, we can come up with
Speaker 1: strategies that help people make better choices and have a
Speaker 1: healthier life, and also provide a healthier life for their children.
Speaker 1: In the consumer world, companies like Toyota are using eye
Speaker 1: tracking technology to measure how effective their show rooms are
Speaker 1: drawing attention to the features that they want to communicate
Speaker 1: to potential buyers. There might be something on a new
Speaker 1: Toyota model that the company really wants people to pay
Speaker 1: attention to, and they use the eye tracking technology to
Speaker 1: see if it's working, and if it's not working, can
Speaker 1: they adjust their approach to make it more attractive. And
Speaker 1: inside vehicles, we're seeing more eye tracking technology, both as
Speaker 1: a safety measure in the case of systems that monitor
Speaker 1: a driver's wakefulness or attention, and as a user interface.
Speaker 1: There's a Canadian research team that's using eye tracking technology
Speaker 1: to look at the potential for driver distractions. So it's
Speaker 1: monitoring a driver so that if the driver takes his
Speaker 1: or her attention away from the road, Let's say they
Speaker 1: are checking the various instrumentation panels, or maybe they're looking
Speaker 1: at their phone trying to read a respond to a text.
Speaker 1: The system the car would actually become aware of this.
Speaker 1: Maybe it would start to engage some driver assist features,
Speaker 1: maybe even taking over control of the car briefly, or
Speaker 1: it might send an alert to the driver saying, Hey,
Speaker 1: your attention is wandering away from the road. You really
Speaker 1: need to pay more mind to the task at hand.
Speaker 1: On the user interface side, Porsche recently revealed an electric
Speaker 1: car concept called the Mission E. I should really get
Speaker 1: Scott Benjamin in here to talk about this concept car.
Speaker 1: It's kind of neat. It's an electronic card. It's supposed
Speaker 1: to be a sports car that could compete with the
Speaker 1: Tesla sports Car electronic sports car, and these would include
Speaker 1: eye tracking type control systems for the in dashboard systems
Speaker 1: as well as the the entertainment systems, the various instrumentation panels.
Speaker 1: It would be able to detect what you're looking at
Speaker 1: and give you information based upon that. And I imagine
Speaker 1: we'll see even more of this technology built into self
Speaker 1: driving vehicles in the future, when the cars we get
Speaker 1: into become interactive environments that allow us to do all
Speaker 1: sorts of things we couldn't do as human drivers. And again,
Speaker 1: this is just scratching the surface. There are so many
Speaker 1: more potential applications and I'm sure I'm going to visit
Speaker 1: this topic again in future episodes to explore some of them.
Speaker 1: If you can't tell I'm really interested in this subject
Speaker 1: because it opens up new possibilities to interact with technology,
Speaker 1: will similtaneously giving us the chance to learn more about ourselves,
Speaker 1: and I kind of dig that. Before I sign off,
Speaker 1: I have an exciting announcement to make. I have a
Speaker 1: new podcast launching. It's called tech Stuff Daily, and as
Speaker 1: the name applies, it will be all about technology and
Speaker 1: it's also going to publish every day. That's that's what
Speaker 1: the daily part means every day Monday through Friday, and
Speaker 1: each episode will be between four and six minutes long,
Speaker 1: give or take, and we'll explore a tech topic that's
Speaker 1: been rambling around in the news cycle. This is not
Speaker 1: replacing Text Stuff. That show is still going to be
Speaker 1: publishing twice a week as always. It's going to exist
Speaker 1: side by side, but now you'll get even more tech
Speaker 1: goodness by subscribing to tech Stuff Daily. Now, in my
Speaker 1: next episode, i'll chat a bit more about using eye
Speaker 1: tracking technology specifically in the realm of e sports, as
Speaker 1: well as a discussion about the sports in general. I
Speaker 1: look forward to chatting with you guys about it, and
Speaker 1: if you have suggestions for future episodes of Text Stuff,
Speaker 1: please let me know you can write me at tech
Speaker 1: Stuff at how stuff works dot com, or drop me
Speaker 1: a line on Facebook or Twitter. The show's handle is
Speaker 1: tech Stuff h SW. Remember I record new episodes every
Speaker 1: Wednesday and Friday, and you can watch me record them
Speaker 1: live at twitch dot tv slash tech Stuff. Just visit
Speaker 1: that you r L. You'll find the schedule there and
Speaker 1: I'll talk to you again really soon for more on
Speaker 1: this and thousands of other topics because it staff works
Speaker 1: dot com.