Speaker 1: Welcome to text Stuff, a production from I Heart Radio.
Speaker 1: Hey there, and welcome to tech Stuff. I'm your host,
Speaker 1: Jonathan Strickland. I'm an executive producer with I Heart Radio
Speaker 1: and I love all things tech and Back in the
Speaker 1: nineties nineties, when Nirvana was teaching us how to have
Speaker 1: all the fields, virtual reality appeared ready to transform the world.
Speaker 1: The hype for the young technology was reaching peak levels.
Speaker 1: Hollywood made movies that incorporated the idea, suggesting that we
Speaker 1: would soon be able to jump into computer generated worlds
Speaker 1: and do everything from navigate file systems in three dimensions
Speaker 1: exciting I know, to taking off like Superman. But as
Speaker 1: people got a chance to actually try out virtual reality
Speaker 1: set ups in the nineteen nineties, they became disillusioned. The
Speaker 1: tech chnology just couldn't live up to the hype. You
Speaker 1: could get an immersive experience, but the graphics were primitive.
Speaker 1: The headsets were typically so heavy they had to be
Speaker 1: suspended from the ceiling with cables. Your movement was limited,
Speaker 1: and system latency caused a lot of folks to get
Speaker 1: motion sickness while trying one out. Interest in VR plummeted,
Speaker 1: and the industry found itself the proverbial redheaded step child
Speaker 1: of the tech world as well as the investment world.
Speaker 1: A lot of people continued to work on VR hardware
Speaker 1: and software, but it was much more challenging to get funding.
Speaker 1: Progress was slow. It typically meant that you were appropriating
Speaker 1: stuff from other industries like the video game world or
Speaker 1: smartphones in order to get your stuff done. Thirty years later,
Speaker 1: we're seeing something similar with dedicated augmented reality headsets. Magic
Speaker 1: Leap a company that at one point was so exciting
Speaker 1: that it raised around three billion dollars that's billion with
Speaker 1: a B. Much of that money, by the way, was
Speaker 1: raised years before the company had anything significant to show off.
Speaker 1: It seemed to be on a pretty similar path as
Speaker 1: VR back in the nineties, and sure enough now in
Speaker 1: twenty twenty, the company scrambled to secure a three fifty
Speaker 1: million dollar lifeline investment after laying off about one thousand
Speaker 1: workers plus the founder of the company has stepped down.
Speaker 1: Today we're going to explore Magic Leaps story, and while
Speaker 1: like a Leap, there's a rise and a fall, this
Speaker 1: story is not yet over. Magic Leap could potentially recover,
Speaker 1: or they might fade into obscurity, or it may just
Speaker 1: become a company lesser than what it was intended to be.
Speaker 1: But before we get into the history of the company,
Speaker 1: it is good for us to define terms like virtual
Speaker 1: reality and augmented reality, both of which fall into a
Speaker 1: spectrum of technologies that a lot of folks call mixed reality.
Speaker 1: And it all has to do with a combination of
Speaker 1: real world experiences and computer generated experiences and meshing them
Speaker 1: together in some way. So if you think of mixed
Speaker 1: reality as a spectrum, on one extreme end, you've got
Speaker 1: pure virtual reality. In VR, computer is providing the sites
Speaker 1: and sounds, and theoretically it could provide other sensations as well,
Speaker 1: including touch through haptic feedback peripherals, or scent through various
Speaker 1: perfumes or oils and dispensers, though not many VR systems
Speaker 1: go as far as replicating smells, and very few, as
Speaker 1: far as I know, replicate taste. But the idea is
Speaker 1: that the computer generated sensory input is substituting for what
Speaker 1: we would see in our real world. So instead of
Speaker 1: seeing a boring office or a retail space in a
Speaker 1: shopping mall, as was the case when I first experienced
Speaker 1: VR back in the ES, we would be transported into
Speaker 1: say a rainforest, or a pirate ship, or a fantasy realm, or,
Speaker 1: as was my case, a bunch of vector graphics of
Speaker 1: really blocky tero dactyls flying around. Using a control system,
Speaker 1: we can navigate these virtual environments and we can interact
Speaker 1: with them. A true VR experience should be both interactive
Speaker 1: and immersive, and not all VR is created equal. Some
Speaker 1: VR experiences are fairly light touch, with only a bit
Speaker 1: of interactivity and primitive graphics. Typically, the hardware for VR
Speaker 1: includes a head mounted display so that your vision focuses
Speaker 1: only on the world that's created by the computer, and
Speaker 1: some headphones, plus some sort of control system. There may
Speaker 1: also be external cameras to help track your motion and
Speaker 1: translate that into what you experience in the virtual environment,
Speaker 1: giving you a little more nuance to your actions. But
Speaker 1: in the end, the thing to remember is that the
Speaker 1: stuff you are experiencing through your senses is in large
Speaker 1: part computer generated. Augmented reality is closer to the other
Speaker 1: end of the spectrum of mixed reality. With a R,
Speaker 1: you have a view of the real world through some
Speaker 1: sort of display. It could be a head mounted display
Speaker 1: like VR, or it could be something as simple as
Speaker 1: a smartphone or handheld game system. But whatever it is,
Speaker 1: it has to be able to overlay digital information on
Speaker 1: top of your view of the real world. So let's
Speaker 1: imagine a pair of glasses that can display information on
Speaker 1: the inside of the lenses so that you can see it.
Speaker 1: The lenses themselves are transparent, so you are looking at
Speaker 1: the world as if you were just wearing a normal
Speaker 1: pair of glasses. And maybe you put on a pair
Speaker 1: while you're walking in an unfamiliar city, and the glasses
Speaker 1: start to display information on the inside of those lenses
Speaker 1: about the area around you. Maybe they provide you directions
Speaker 1: so you can get to a specific destination. With augmented reality,
Speaker 1: the whole idea is that a computer system isn't replacing
Speaker 1: your real world ex experience, but instead it's enhancing or
Speaker 1: augmenting your experience in the real world. Between true VR
Speaker 1: and a R is a whole band of mixed reality technologies.
Speaker 1: Some are headsets allow wearers to have experiences in which
Speaker 1: the display they use inserts visual elements into their physical space.
Speaker 1: So you could wear a mixed reality headset, look at
Speaker 1: the room around you and watch as a video game
Speaker 1: character like I don't know, like Mario or Master chief
Speaker 1: or crash Bandicoot just walks right into the room. A
Speaker 1: tabletop might become a virtual battleground for a little toy armies.
Speaker 1: A wall could become a window into a fantasy realm.
Speaker 1: The possibilities are limited only by computer processors, the software,
Speaker 1: and the imagination of developers. VR and a R share
Speaker 1: a lot of history, and a lot of tech engineers
Speaker 1: have repurposed technology intended for one and used it for
Speaker 1: the other. And I think it's safe to say that most,
Speaker 1: but not all VR implementations have a head mounted display
Speaker 1: that can track where someone is looking and then adjust
Speaker 1: the point of view in the virtual environment accordingly. So
Speaker 1: that way, if you turn your head left, your virtual
Speaker 1: point of view also turns left. And that's incredibly important
Speaker 1: if you want an immersive experience. Just imagine how disorienting
Speaker 1: it would be if you put on a headset and
Speaker 1: you're looking at a video game world and you turn
Speaker 1: your head to look up at the sky, but your
Speaker 1: view looks down at your feet. That would really throw
Speaker 1: you off. Likewise, not all a OUR implementations have a
Speaker 1: head mounted display, but many do. These displays have to
Speaker 1: do something a little different from VR. They have to
Speaker 1: present a view of the real world. Now, they either
Speaker 1: do that through a transparent view screen or through a
Speaker 1: video screen that's got a video feed of the real world.
Speaker 1: So in other words, that screen has to be paired
Speaker 1: with a camera that's feeding a live image to the screen.
Speaker 1: To do that, thee has to be able to analyze
Speaker 1: what someone is looking at in order to overlay relevant information. Right,
Speaker 1: it doesn't do any good if you just get random information.
Speaker 1: I mean, if you were to wear an a R
Speaker 1: headset and you're walking through Paris, but then the headset
Speaker 1: starts to giving you directions as if you're on a
Speaker 1: walk in Moscow, that would be less than useless. So
Speaker 1: the tech needs sophisticated code to interpret information. That could
Speaker 1: include things like image recognition, compass directions, GPS, coordinates, your orientation,
Speaker 1: and more. All of that is super complicated, and it
Speaker 1: goes a bit beyond our story, but it's a good
Speaker 1: thing to remember that to pull off either convincing VR
Speaker 1: or a really useful a R experience, you need a
Speaker 1: lot of sophisticated code, and typically you need a fairly
Speaker 1: powerful computer system running in the background, whether it's on
Speaker 1: board in some sort of device you're wearing or tethered
Speaker 1: to a smartphone or tethered to a PC. VR and
Speaker 1: a R are not interchangeable, but they are close cousins.
Speaker 1: Augmented reality is probably the more productive member of the family.
Speaker 1: Many but not all, of virtual realities applications relate to
Speaker 1: entertainment in some way, and many but not all of
Speaker 1: augmented realities applications relate to productivity. Okay, that's a good
Speaker 1: overview of two of the big subcategories of mixed reality.
Speaker 1: Though I should point out there are many other aspects
Speaker 1: of mixed reality that I've not covered, but that could
Speaker 1: be its own episode. Now let's move on to Magic Leap.
Speaker 1: The company's founder is Ronnie Abovitz. Abovitz was born in Ohio.
Speaker 1: His family moved to Florida when he was a kid,
Speaker 1: and he went to the University of Miami, where he
Speaker 1: ultimately earned a master's degree in biomedical engineering. So I
Speaker 1: think it's safe to say he is pretty darn sharp.
Speaker 1: He earned his master's in nine and the following year
Speaker 1: he would co found his first company, e CAT that's
Speaker 1: a Z hyphen k a t Z. CAT was a
Speaker 1: medical technology company. The focus of the company was to
Speaker 1: develop technologies to assist in minimally invasive surgical procedures, which
Speaker 1: have numerous benefits including a reduced risk of infection and
Speaker 1: faster recovery times. The company developed robotic tools that surgeons
Speaker 1: could control to make precise, delicate incisions. The control systems
Speaker 1: could translate surgeon commands into steady and small motions with
Speaker 1: the robotic arm removing jitter and other muscular movements that
Speaker 1: would make these kind of procedures difficult or impossible with
Speaker 1: that level of precision. In two thousand four, the company
Speaker 1: chose to spin off its robotics division into a new
Speaker 1: entity called Mako Surgical Corporation. And the logo for Mako
Speaker 1: was super cool too, reminiscent of a shark's fin makos
Speaker 1: a type of shark, and as a side note, the
Speaker 1: Magic Leap logo is reminiscent of a leaping whale, So
Speaker 1: there's a very marine theme going with Abovitz's work. So
Speaker 1: Abavits served not just as a co founder of Mako,
Speaker 1: but the chief technology officer and chief visionary officer for
Speaker 1: the company. Like z Cat, Mako would develop technologies to
Speaker 1: help surgeons as well as implants for stuff like knee
Speaker 1: replacement surgery. The MAKO systems gave surgeons the information they
Speaker 1: needed to make real time decisions during a surgical procedure,
Speaker 1: including feedback systems to alert surgeons when they were getting
Speaker 1: close to the border of a predefined safety zone for
Speaker 1: a particular procedure. It's all super cool stuff that helps
Speaker 1: minimize any impact to quality of life during the recovery period.
Speaker 1: Sometime around twenty Orlen Abovitz began to think about another project.
Speaker 1: The surgical assistance technologies rely partly on augmented reality tech.
Speaker 1: Digital information would augment the surgeons operating room, but A
Speaker 1: Pivitz saw possible applications for this kind of technology that
Speaker 1: extended not used into the medical field, but beyond. He
Speaker 1: called the project Magic Leap, and he would create a
Speaker 1: corporation under that name INN. But as for the details
Speaker 1: of what Magic Leap was all about, that remained a mystery.
Speaker 1: Things didn't exactly coalesce the following year when Abavits put
Speaker 1: together a presentation for the two thousand twelve ted X
Speaker 1: event in Saratoga, Florida. The presentation is on YouTube and
Speaker 1: it's really more performance art than anything else. It actually
Speaker 1: reminds me a lot of the interactive art experiences from
Speaker 1: meal Wolf, a collection of artists out of Santa Fe,
Speaker 1: New Mexico, and perhaps by no coincidence, Meal Wolf is
Speaker 1: listed as a potential artistic partner with Magic Leap. By
Speaker 1: the way, if you're interested in art, performance, tech, and absurdism,
Speaker 1: I highly recommend you check out Meal Wolf. Their work
Speaker 1: ranges from whimsical to unsettling. Anyway, I don't think I
Speaker 1: could describe the ted X presentation from two thousand twelve effectively.
Speaker 1: It's just kind of weird. If you want to check
Speaker 1: it out. It is on YouTube and it's called the
Speaker 1: Synthesis of Imagination. And I'll just say this, there is
Speaker 1: no hint at all what the heck Magic Leap is
Speaker 1: about in that video. Now, that was sort of okay.
Speaker 1: At that point, the company name wasn't widely known, and
Speaker 1: you would be forgiven if you were walking away thinking
Speaker 1: magically was some sort of art collective that was particularly
Speaker 1: obsessed with fudge. Seriously, you need to watch the ted
Speaker 1: X talk from two thousand twelve to understand that reference.
Speaker 1: The team began working on a longer term goal of
Speaker 1: creating a mixed reality system capable of generating really compelling,
Speaker 1: impressive digital overlays on top of our real world, not
Speaker 1: even just on top of incorporated into our real world.
Speaker 1: In two thousand fourteen, more people would begin to take
Speaker 1: note of Magic Leap as it held a round of
Speaker 1: investment funding and brought in fifty million dollars in the
Speaker 1: first quarter of the year, But Magic Leap had yet
Speaker 1: to get truly widespread fame at that point. That did
Speaker 1: begin to change once people heard who some of these
Speaker 1: investors were. One of them was Richard Taylor, the Oscar
Speaker 1: winning special effects guru who worked on movies like Lord
Speaker 1: of the Rings and Peter Jackson's King Kong. Other prominent
Speaker 1: figures in fields like video games and tech companies also
Speaker 1: were among the number of investors, and in October two
Speaker 1: thousand fourteen, Magic Leap received five hundred forty two million
Speaker 1: dollars from numerous investors, including Google. It also didn't hurt
Speaker 1: that Abavits had recently seen his company Mako get acquired
Speaker 1: for a significant sum of money, more than a billion dollars,
Speaker 1: so things were going pretty fast and furious at this point. Now,
Speaker 1: clearly something special had to be happening, right, But what
Speaker 1: was it? As we'll see the mystery would be part
Speaker 1: of the selling point and the problem for Magic Leap.
Speaker 1: But before we get to that, let's take a quick break.
Speaker 1: Abevids and crew made a choice to be secretive and
Speaker 1: mysterious with the goal of the company, which did drum
Speaker 1: up a lot of speculation and buzz about Magic Leap.
Speaker 1: But that can be a double edged sword, because while
Speaker 1: people were starting to talk about this enigmatic company, it
Speaker 1: also meant that they were left to fill in the
Speaker 1: gaps of their knowledge by themselves. And here's a little
Speaker 1: secret for all of you out there. The odds are
Speaker 1: that whatever thing you're really working on is unlikely to
Speaker 1: match up to unfettered imagination. And this is why horror
Speaker 1: movies that leave stuff up to the imagination of the
Speaker 1: audience can have a much longer lasting effect than a
Speaker 1: movie that just shows you everything. The same is true
Speaker 1: for technology. If you don't lay out your vision, people
Speaker 1: kind of do that for you. And if that vision
Speaker 1: doesn't line up with reality, you're setting yourself up for failure.
Speaker 1: As you know, you're bound to disappoint people in the end.
Speaker 1: But maybe I'm jumping ahead too much here. The company
Speaker 1: did give a few members of the media some early
Speaker 1: glimpses into technology it was working on. One of those
Speaker 1: journalists was a New York Times reporter named John Markoff.
Speaker 1: Magically demonstrated some early technology to mark Off, not a
Speaker 1: production model, just something that was sort of in the
Speaker 1: R and D department. This implementation wasn't a wearable headset.
Speaker 1: Markoff described it as looking like quote, something from an
Speaker 1: optometrist's office end quote, And it was projecting what is
Speaker 1: called a digital light field into his eyes. So what
Speaker 1: the heck does that mean and why is it necessary
Speaker 1: at all? Well, Markoff goes into the fact that one
Speaker 1: of the challenges with both VR and a R has
Speaker 1: to do with depth and the way we perceive depth.
Speaker 1: So let's say you're looking down the street and you're
Speaker 1: watching a someone. Let's say it's Josh Clark, of stuff,
Speaker 1: you should know someone this Josh Clark. He's skipping merrily
Speaker 1: down the street towards you. And actually, yeah, that's that's
Speaker 1: actually how he walks, and no, I don't know why
Speaker 1: he does that. Anyway, you would see Josh Clark is
Speaker 1: starting off pretty far away from you. You'd focus on
Speaker 1: him in the distance, but then he's gradually getting closer
Speaker 1: and closer, skipping, perhaps somewhat menacingly toward you. He would
Speaker 1: appear to get larger because he's getting closer, until at
Speaker 1: some point he would block out your vision due to
Speaker 1: the fact that he has also and he does not
Speaker 1: mention this very much. He's also a giant. Okay, but
Speaker 1: let's say you're sitting with a VR headset on instead,
Speaker 1: and this is simulating you looking down the street. Now
Speaker 1: you see a virtual representation of Josh Clark skipping toward you,
Speaker 1: but you're not actually looking at someone who is really
Speaker 1: far away. Right, the virtual Josh Clark is an image
Speaker 1: on a screen, and that screen is just an inch
Speaker 1: or two away from your eyes. Josh isn't actually getting
Speaker 1: closer to you. Instead, the image is getting bigger. To
Speaker 1: simulate that sensation, you're getting visual cues that he's getting closer,
Speaker 1: but the entire time you're focusing on something that's actually
Speaker 1: right in front of you that screen. Now, it's hard
Speaker 1: to create a real sense of depth on a screen.
Speaker 1: You can create visual cues to simulate depth, mimicking what
Speaker 1: we perceive in the real world, but then we have
Speaker 1: a situation in which our eyes are trying to focus
Speaker 1: on something that in our brains appears to be very
Speaker 1: far away from us, but in fact it's on a
Speaker 1: screen that's right in front of us. This can create
Speaker 1: fatigue and discomfort for the viewer, and it's called old
Speaker 1: virgins accommodation conflict. And if we move this over to
Speaker 1: augmented reality, we can really see a disconnect. The physical
Speaker 1: world around us has actual depth. If we want a
Speaker 1: compelling a our experience that is more than just projecting
Speaker 1: text in front of our eyes, we need a system
Speaker 1: that can also incorporate depth in our virtual images so
Speaker 1: that they fit into the real world around us. Otherwise
Speaker 1: we'll be trying to focus on stuff that's on a screen,
Speaker 1: even if it's a transparent one that's right in front
Speaker 1: of us, while also trying to focus beyond in the
Speaker 1: real space around us. Enter the light field. Now, way
Speaker 1: back in the day, I talked about light field technology
Speaker 1: with the lightro camera. This digital camera could do something
Speaker 1: super cool. You could take a digital photo with it
Speaker 1: and then after the fact, you could change the focus
Speaker 1: of that image. Now, to understand why that's cool, let's
Speaker 1: think about standard photography. A camera, whether film or digital,
Speaker 1: has a lens. Typically there are cameras that don't have lenses,
Speaker 1: but let's just talk about the lensed ones. The purpose
Speaker 1: of the lens is to direct light. It bends the
Speaker 1: light toward a photoreactive element, either photoreactive film or a
Speaker 1: digital sensor. The physical curvature of the lens will determine
Speaker 1: how much it will bend that light. By moving the
Speaker 1: lens closer to or further from that photoreactive element, you
Speaker 1: can direct light from different distances toward that element. You
Speaker 1: will get a different point of focus, a different focal point.
Speaker 1: So you might have one setting that you can capture
Speaker 1: so that you get a clear image of a very
Speaker 1: distant object, and then you would have a totally different
Speaker 1: setting if you wanted to get a clear photo of
Speaker 1: something that was much closer to you, or you might
Speaker 1: even swap out lenses if you had to get a
Speaker 1: wider angle, or if you wanted to really zoom into
Speaker 1: something that was really further away. But the nitro camera
Speaker 1: and other light field cameras are different. They capture a
Speaker 1: light field, which describes all the rays of light moving
Speaker 1: in all directions within a given area the frame of
Speaker 1: view of the camera, and it includes not just the
Speaker 1: direction of the light, but the intensity of each ray
Speaker 1: of light. The special images these cameras produce, which require
Speaker 1: a special app to manipulate afterward, represent all the different
Speaker 1: rays of light within a shot, and so you can
Speaker 1: change the focus of that image. You can choose which
Speaker 1: rays of light you want to really focus on. So
Speaker 1: if you've got one object that's close to you and
Speaker 1: another that's in the background, you can actually switch focus
Speaker 1: from background to foreground over and over again, just choosing
Speaker 1: whichever one you want to look at at a given time.
Speaker 1: But how the heck does this technology actually work. It's
Speaker 1: one thing to say that the camera captures the light
Speaker 1: field of a scene, and another to actually make something
Speaker 1: that can do at So, your typical light field camera
Speaker 1: has all the elements of a normal camera. That is,
Speaker 1: has a lens to direct incoming light. There's an aperture
Speaker 1: or a hole that the light must pass through in
Speaker 1: order to hit a sensor for recording the image. But
Speaker 1: the lightro also has what's called a micro lens array,
Speaker 1: which sits between the main lens and the sensor. Now,
Speaker 1: as that name suggests, a micro lens array is a
Speaker 1: collection of very small lenses that can capture light coming
Speaker 1: from different distances from the camera's sensor. If you think
Speaker 1: of a camera as having detachable lenses where you can
Speaker 1: put a different lens on in order to get a
Speaker 1: different focal point. Just imagine a whole sheet of tiny
Speaker 1: little lenses that all can do that, and each lens
Speaker 1: has a slightly different curve to it so it can
Speaker 1: focus at a slightly different distance. You've got an entire
Speaker 1: array of these tiny lenses and they're all directing light
Speaker 1: toward particular pixels the digital cameras sensor. Each micro lens
Speaker 1: is doing the same thing and for its own focal
Speaker 1: point that is. That means you can have different micro
Speaker 1: lenses dedicated to those specific focal points, all capturing all
Speaker 1: the information to that sensor. Using the special app and
Speaker 1: photo file type, you can then select which batch of
Speaker 1: information you want to look at, and in this case,
Speaker 1: the batches of information relate to different focal points. So
Speaker 1: you can say, I want to look at all the
Speaker 1: light that was at this object, and it brings that
Speaker 1: object into focus. Then you click on a different object
Speaker 1: and you say, now I want to look at that one,
Speaker 1: and it brings all that light into focus instead. It's
Speaker 1: just switching lenses virtually in a way. There are companies
Speaker 1: working on technologies that will allow for light field displays
Speaker 1: without a micro lens array, but that topic is pretty
Speaker 1: darn complicated, even more so than just the standard light
Speaker 1: field cameras, So I'm gonna save that for another time. Now.
Speaker 1: The magic Leap concept that Mark of got to see
Speaker 1: also had light field technology, but in this case it
Speaker 1: was a light field display, and the goal was to
Speaker 1: project an image onto the retina of the user's eye,
Speaker 1: rather than to create a special type of digital photo. Ideally,
Speaker 1: the goal would be to create three dimensional virtual objects
Speaker 1: using depth cues and meshing those virtual objects with the
Speaker 1: real environment, so that to your eye it would be
Speaker 1: no different than if the virtual object really were there
Speaker 1: in the same space as you. So if you had
Speaker 1: a floating castle as your virtual object, to your eye,
Speaker 1: it should look like the floating castle really is floating there.
Speaker 1: Not only would the virtual objects seem to belong to
Speaker 1: the physical space that appeared to occupy. You could view
Speaker 1: that virtual stuff from different angles. So you're floating castle,
Speaker 1: you could walk around it, and you let's say it's
Speaker 1: floating at say chest level, you could walk around see
Speaker 1: it from every single angle. It would be sitting there
Speaker 1: in the air as if it were actually be there.
Speaker 1: You could squat down and look under it. You could
Speaker 1: get on your tiptoes and look at the turrets and
Speaker 1: the parapets and everything. From the top. It would appear
Speaker 1: to actually have a physical presence. Now, markov only got
Speaker 1: a first glimpse at this technology, and to be fair,
Speaker 1: the New York Times piece was really more about the
Speaker 1: general potential of a R and VR and light field technologies.
Speaker 1: But that piece did have a little nugget from Abovits,
Speaker 1: who said, quote, our real market is people doing everyday things.
Speaker 1: Rather than pulling your mobile phone in and out of
Speaker 1: your pocket, we want to create an all day flow.
Speaker 1: Whether you're going to the doctor or a meeting or
Speaker 1: hanging out, you will all of a sudden be amplified
Speaker 1: by the collective knowledge that is on the web end quote. Now,
Speaker 1: I think the phrase that will end up being something
Speaker 1: we'll have to look back on is our real market
Speaker 1: is people doing everyday things. I've got to keep that
Speaker 1: in mind when we get to stuff like how much
Speaker 1: the actual product costs. Now the world got a little
Speaker 1: more insight into what Magic Leap wanted to do. They
Speaker 1: wanted to create an a R technology that could enhance
Speaker 1: our day to day activities in various ways. But while
Speaker 1: there were some tentalizing details fed to the general public
Speaker 1: a lot of Magic Leaps actual work was still an enigma.
Speaker 1: In two thousand and fifteen, Magically began releasing demo videos
Speaker 1: of what its technology would be able to do. These
Speaker 1: videos stated that the footage that was shot was through
Speaker 1: the Magic Leap technology, with no special effects or compositing
Speaker 1: used after the fact. The demos were definitely compelling. My
Speaker 1: favorite one showed a miniature virtual representation of the Solar
Speaker 1: system seemingly hanging in the air over a desk, and
Speaker 1: right next to the solar system was a woman at
Speaker 1: work on a computer. The details of the model were incredible.
Speaker 1: They included a little rotating Earth, and they were all
Speaker 1: had incredibly vibrant colors. It was amazing. The Sun, the planets,
Speaker 1: the Moon's all looked solid as they moved in their pathways.
Speaker 1: Really breathtaking stuff and those videos are still up on
Speaker 1: YouTube if you want to check them out. The company
Speaker 1: filed for a patent on its technology, which included a
Speaker 1: mention of a quote photonic light field chip in the quote,
Speaker 1: presumably the technology that could create the projection of virtual
Speaker 1: objects to then be beamed right into a person's eyeballs.
Speaker 1: The patent also called for quote an inertial measurement unit
Speaker 1: in the quote or I am you. This would later
Speaker 1: be revealed to be a device that would hold the processor,
Speaker 1: the storage, and the battery for the Magic Leap system.
Speaker 1: It would be something that you would have to you know,
Speaker 1: wear on a belt, you know, clip it to a pocket,
Speaker 1: something like that. And offloading the tech onto a tethered,
Speaker 1: wearable pack meant that the engineers didn't have to figure
Speaker 1: out how to cram all those components into the form
Speaker 1: factor of the headset itself, which would help keep the
Speaker 1: glasses more lightweight and less bulky. Also, it would help
Speaker 1: deal with heating issues. I mean, this thing was going
Speaker 1: to require a lot of processing, and processors can run
Speaker 1: a little warm, and you don't want to have a
Speaker 1: super hot processor right next to, say, your temple. Now,
Speaker 1: the mysterious nature of Magically didn't hurt when it came
Speaker 1: to investments. Heck, maybe maybe it actually helped a lot.
Speaker 1: Investors convinced that this could be the next big thing
Speaker 1: poured cash into magic Leap. In early twenty sixteen, the
Speaker 1: company had already raised nearly one point four billion dollars
Speaker 1: in investments. Abavits had a great reputation, his work in
Speaker 1: medical robotics had been a proven success, and while there
Speaker 1: had been other attempts at creating augmented reality or mixed
Speaker 1: reality headsets, no one had really nailed it yet. Magically
Speaker 1: claimed that the company was working on a totally new
Speaker 1: approach to generating augmented reality experiences, which meant that maybe
Speaker 1: it was sidestepping some of the limitations that had held
Speaker 1: back other implementations. So instead of saying, oh, well, we
Speaker 1: we're going to get to market after Microsoft and the
Speaker 1: hollow lens, they could say we're doing a totally different
Speaker 1: approach to augmented reality that is unlike what they are doing. Slowly,
Speaker 1: the potential for magic Leap appeared to be forming in
Speaker 1: the minds of the public, and this could be way
Speaker 1: more advanced than a system that would let you play games.
Speaker 1: You could use it to replace stuff like televisions or
Speaker 1: computer displays. You could create virtual screens in your environment.
Speaker 1: The device would remember where you put stuff, so if
Speaker 1: you designated a segment of your wall as a giant television.
Speaker 1: That's where it would appear. Whenever you wore the Magic Leap,
Speaker 1: you could still look around and the screen would appear
Speaker 1: to be anchored in that physical space, as if you
Speaker 1: had actually hung a TV there. So with this device,
Speaker 1: you could have multiple computer screens open, all at the
Speaker 1: same time, and none of them would take up any
Speaker 1: actual physical space. Now, when we come back, we'll talk
Speaker 1: about the launch of the real Magic Leap one headset,
Speaker 1: the tech that makes it work, and what has happened
Speaker 1: to the company since that product launch. But first, let's
Speaker 1: take another quick break. Magic Leap got its start into
Speaker 1: two thousand eleven, it would not have a product to
Speaker 1: launch for nearly a full decade. Before that, the company
Speaker 1: did offer up a limited developer edition of the technology
Speaker 1: called the Magic Leap one Creator's Edition, starting in August
Speaker 1: two thousand eighteen. It would cost two thousand, two hundred
Speaker 1: nine five dollars, a princely sum. Indeed, a journal called
Speaker 1: The Information would later allege that Magic Leap was only
Speaker 1: able to sell six thousand of the Creator Edition model
Speaker 1: upon the first six months of its release, and it
Speaker 1: also said that Magic Leap's goal was to sell one
Speaker 1: hundred thousand units, so the goal was one hundred thousand.
Speaker 1: Real was six thousand, so as you can imagine, that
Speaker 1: represents a pretty severe shortfall. And between the launch of
Speaker 1: the Creator addition in two thousand eighteen and the actual
Speaker 1: Magic Leap one headset in December two thousand nineteen, the
Speaker 1: company would see cutbacks. It would have a travel freeze
Speaker 1: so that it wasn't gonna pay for corporate travel. There
Speaker 1: were layoffs, There were a lot of departures. Beyond the layoffs,
Speaker 1: the shine on the company was starting to dull a bit,
Speaker 1: and it had been a long time since those early
Speaker 1: demo videos and the market response to the Creator edition
Speaker 1: was underwhelming to say the least. No doubt, some of
Speaker 1: the investors were starting to get a bit an see.
Speaker 1: After all, more than two billion dollars had already gone
Speaker 1: to the company since it got started. Now it was
Speaker 1: in December two thousand nineteen that Magic Leap introduced the
Speaker 1: Magic Leap one system. The company had only held a
Speaker 1: few demonstrations, mostly to select members of the press, not
Speaker 1: a whole lot of public demos of this technology, and
Speaker 1: the demo and concept videos had impressed a lot of people.
Speaker 1: A few years before. One famous concept video showed a
Speaker 1: school gym filled with students that were sitting on the bleachers.
Speaker 1: They were looking into an otherwise empty gym, and suddenly
Speaker 1: a whale appears to breach from the gym floor, leaping
Speaker 1: high into the air and the kids all gasp. But
Speaker 1: was that the sort of experience people could actually come
Speaker 1: to expect with the Magic Leap One. Well maybe, but
Speaker 1: first you would have to meet that sales price, which
Speaker 1: was the same as the Creator Edition, a hefty two
Speaker 1: thousand two dollars. That's still the price for the spatial
Speaker 1: computer as of this recording. And that includes the headset,
Speaker 1: which is officially known as Lightweare and the I Am
Speaker 1: You now known as the Light Pack and that houses
Speaker 1: the processors and battery for the Magic Leap. It's connected
Speaker 1: to the Magic Leap headset with a wire and it
Speaker 1: also includes a wireless controller that you can hold in
Speaker 1: one hand, similar to the we nun chuck in a way.
Speaker 1: The processor in the Magic Leap one is an Nvidio
Speaker 1: Parker s o C. S o C stands for System
Speaker 1: on a chip. It has six processor cores. There's also
Speaker 1: an Nvidio Pascal graphics processing unit. Or GPU, and it
Speaker 1: has eight gigabytes of ram uh. It also has a
Speaker 1: one eight gigabyte hard drive, although only ninety five gigabytes
Speaker 1: of actual storage space is available on it. The battery
Speaker 1: in the light pack is said to hold enough juice
Speaker 1: for three hours of use, or you can actually plug
Speaker 1: into an a wall. You have an little a C
Speaker 1: adapter that you can plug into a wall for sustained use,
Speaker 1: though obviously if you do that you're tethered to a
Speaker 1: power outlet. One thing Magic Leap does differently from other
Speaker 1: devices is that the processing can all take place on
Speaker 1: the system itself, unlike, for example, Google Glass, which you
Speaker 1: would have to pair with a smartphone. So Google Glass
Speaker 1: is more of a smartphone peripheral, whereas Magical Leap one
Speaker 1: is a computer system all by itself. The light pack
Speaker 1: can connect via USB or WiFi to a computer and
Speaker 1: you can use that to download or launch apps. According
Speaker 1: to Omar Khan, the product officer for a Magic Leap,
Speaker 1: the Magical Lea one device had some minor updates compared
Speaker 1: to the original Creator edition, but at least on casual glance,
Speaker 1: the two seem pretty close to identical. The headset, which
Speaker 1: looks kind of like goggles, has speakers built into them
Speaker 1: and that can create immersive directional sounds. So if you
Speaker 1: wore the Magic Leap and you were playing a game,
Speaker 1: you would be able to hear virtual things moving around
Speaker 1: your real environment. You might be able to hear something
Speaker 1: sneaking up on you. You'd turn around, you would actually
Speaker 1: see it there. That's pretty darn cool. It also has
Speaker 1: a couple of infrared emitters and sensors built into the
Speaker 1: front of the headset, and these are used to create
Speaker 1: a digital map of your physical environment. See to make
Speaker 1: a virtual character appear as if it is actually in
Speaker 1: a physical space. First, the Magical Leap one has to
Speaker 1: know what your physical space is like. So those emitters
Speaker 1: send out pulses of infrared light, and those pulses reflect
Speaker 1: off the various surfaces in your space, and then they
Speaker 1: come back and get picked up by the infrared sensors
Speaker 1: on the front of the Magic Leap one. It's a
Speaker 1: little bit like echolocation, and it gives the Magic Leap
Speaker 1: one processor the data it needs to understand depth in
Speaker 1: your environment. Measuring the time it takes the infrared to
Speaker 1: go out and come back tells the Magic Leap if
Speaker 1: a table is closer to you than a chair, for example,
Speaker 1: and so when the Leap starts to project a virtual
Speaker 1: image into your view, it can incorporate the physical environment
Speaker 1: more convincingly. The controller has a magnetic controller tracking device
Speaker 1: in it. The controller has an emitter, the headset has
Speaker 1: a receiver, and the controller allows for all sorts of
Speaker 1: interactions within a virtual environment, from playing a game to
Speaker 1: performing simple controls with a virtual video screen. The magically
Speaker 1: one also can track hand movements, so you can do
Speaker 1: gesture base controls as well. So it all really depends
Speaker 1: on the application and what you want to do. The
Speaker 1: real star of the show would be the light field.
Speaker 1: Projectors six small lenses lit from the side angle light
Speaker 1: to hit your eyes, and it's really no different than
Speaker 1: you looking at something without glasses on at all. Right,
Speaker 1: light from the environment you are in hits your eyes
Speaker 1: and then your brain susses out what you're looking at.
Speaker 1: With the Magic Leap one, the light isn't coming just
Speaker 1: from the environment, it's also coming from the device. But
Speaker 1: if it's done well, your brain doesn't necessarily register the difference,
Speaker 1: and so to you it appears as though these virtual
Speaker 1: elements are really in your physical space. The headset does
Speaker 1: restrict your field of view. However, the visual field refers
Speaker 1: to the area of space a person can see at
Speaker 1: any given moment. We measure it by degrees from the center,
Speaker 1: so this includes the area you focus on as well
Speaker 1: as your peripheral vision. So a typical visual field has
Speaker 1: a full range of one fifty five degrees across horizontally
Speaker 1: and one thirty five degrees up and down or vertically.
Speaker 1: The magically one, in contrast, has a horizontal field of
Speaker 1: view of forty degrees and a vertical field of view
Speaker 1: of thirty degrees, So clearly it's much more restricted than
Speaker 1: the typical one by one thirty five. A lot of
Speaker 1: that unrestricted view is peripheral, so it's stuff you're not
Speaker 1: necessarily really focusing on, but you still can see it.
Speaker 1: So wearing a magically one means that you see less
Speaker 1: of your surroundings. However, you could argue it also helps
Speaker 1: direct your focus towards the virtual elements, so perhaps it's
Speaker 1: also boosting the effect of having those virtual elements in
Speaker 1: your environment. Now, I've not tried these on myself, so
Speaker 1: I can't speak as to what it's like to actually
Speaker 1: wear them. That being said, the reports I've read make
Speaker 1: it sound pretty darn amazing, But at that price, the
Speaker 1: Magic Leap was not really in the ballpark for a
Speaker 1: consumer device. Most folks don't dish out two thousand dollars
Speaker 1: for a fully fledged PC. I mean, the gamers do,
Speaker 1: but most of us don't. So Magic Leap was gearing
Speaker 1: their marketing efforts afterward towards enterprises, much as Microsoft had
Speaker 1: with the Hollow Lens device, another augmented reality headset. They
Speaker 1: did this pretty late in the game, though, so this
Speaker 1: is another point of criticism people have levied at Magic
Speaker 1: Leap because so much of their marketing early on appeared
Speaker 1: to be directed toward the average person. I mean, this
Speaker 1: was supposed to help us in our day to day activities, right,
Speaker 1: But now it was becoming clear that just the cost
Speaker 1: of producing this thing meant there was no way to
Speaker 1: make it affordable for a consumer market. You're going to
Speaker 1: have to market it towards the enterprise. But that was
Speaker 1: a disconnect between the earlier messaging from Magic Leap and
Speaker 1: that report I mentioned from the information was pretty brutal.
Speaker 1: It painted a picture of a company that had failed
Speaker 1: to meet both external and internal expectations worse. The piece
Speaker 1: suggested that a second generation of the technology, a Magic
Speaker 1: Leap to perhaps one at a lower price point, was
Speaker 1: at least two years further down the road. And then
Speaker 1: twenty happened, and it's still happening. It's happened like crazy
Speaker 1: to me, and I'm one of the ones who's lucky
Speaker 1: that it only really happened a little bit. I hope
Speaker 1: the happening to you has been as smooth as possible.
Speaker 1: But man, we're all gonna need to do over, I think. Anyway,
Speaker 1: Magic Leap was already on shaky ground. The lackluster response
Speaker 1: to the Creator Edition was a bad blow. The informations
Speaker 1: report on the company wasn't much better. So Magically collected
Speaker 1: a veritable who's who in speculative fiction, special effects, computing
Speaker 1: and more, all of them becoming part of the co
Speaker 1: report officers or board of directors, really elevating the perception
Speaker 1: of Magic Leap. But that collection of luminaries began to dwindle.
Speaker 1: Some were leaving on their own accord, some left after
Speaker 1: offices shut down, and fewer and fewer of those big
Speaker 1: names were still with the company. One person who stepped
Speaker 1: down was Abo, its himself, who relinquished his position as
Speaker 1: head of the company in March twenty. That was around
Speaker 1: the same time that the company announced the one thousand
Speaker 1: employee layoffs ABO its remains connected to the board of directors. However,
Speaker 1: so what exactly happened, Well, I think in a way
Speaker 1: we saw a repeat of the boom and bust of
Speaker 1: virtual reality back in the nineteen nineties. There was a
Speaker 1: hype train that inflated interest. I don't know what the
Speaker 1: corporate culture of Magic Leap is like. I don't know
Speaker 1: if the company indulged in extravagance the way a lot
Speaker 1: of startups have in the past. I haven't seen reports
Speaker 1: of lavish parties or ridiculous expense says, but just developing
Speaker 1: that technology and getting it into a form factor that's
Speaker 1: small enough to fit into goggles and a puck sized
Speaker 1: clip on device, just that is an enormous challenge. Developing
Speaker 1: the tech had to be expensive, and getting the involvement
Speaker 1: of various big thinkers like Neil Stevenson snow Crash himself,
Speaker 1: that had to cost a pretty penny as well. And
Speaker 1: then you've got a lot of money going out and
Speaker 1: not a lot coming in. You can see that things
Speaker 1: can't go like that forever. Now. According to the Verge,
Speaker 1: Magic Leap is dealing with some pretty tricky issues. Enterprises
Speaker 1: aren't exactly champing at the bit to invest inexpensive experimental hardware,
Speaker 1: particularly hardware that is designed to run apps that can
Speaker 1: only come from a single, unproven source. You know, just
Speaker 1: as Apple controls what apps you can run on an
Speaker 1: iPhone by vetting them before they can go into the
Speaker 1: app store, Magic Leap was making itself the one source
Speaker 1: for software to run on the hardware. Developers would have
Speaker 1: to submit software to Magic Leap for it to become
Speaker 1: available to run on Magic Leap headsets. But if companies
Speaker 1: aren't confident that Magic Leap is going to stick around,
Speaker 1: it makes no sense to sink thousands of dollars in
Speaker 1: hardware that isn't able to run other types of applications. Really,
Speaker 1: what this does is drive home how hard it is
Speaker 1: to do augmented reality well or mixed reality, if you prefer.
Speaker 1: Most companies have to make compromises. You can reduce the
Speaker 1: cost of hardware by offloading work onto a different device
Speaker 1: like a smartphone or a computer, But if you're using
Speaker 1: a phone, then you're gonna have limits to what you
Speaker 1: can accomplish just based on that phone's processor and connection speed.
Speaker 1: If you're using a computer, you have to figure out
Speaker 1: how are you transmitting data between the computer and your device.
Speaker 1: It might limit the uses for your tech because you
Speaker 1: may not be able to move that far away from
Speaker 1: the home base computer. Magic Leap was trying to create
Speaker 1: an advice that wouldn't depend on any other gadget, avoiding
Speaker 1: that kind of compromise, but that also meant the cost
Speaker 1: for the device was going to be much higher to
Speaker 1: cover the cost of all those components. On top of that,
Speaker 1: the company was really trying to make most of those
Speaker 1: components itself rather than rely on existing third party hardware.
Speaker 1: That would just add to the cost because you have
Speaker 1: to develop all those pieces. In fact, it's quite possible
Speaker 1: that originally they were looking to develop their own processors
Speaker 1: before they moved to the Vidio ones. In short, there
Speaker 1: were a lot of very lofty expectations for the technology,
Speaker 1: and even if the technology delivers upon the experience, the
Speaker 1: expense means that most of us don't get an opportunity
Speaker 1: to try it out, and that's unfortunate. There are a
Speaker 1: lot of questions out there over whether or not Magic
Speaker 1: Leap will be able to recover from this pandemic is
Speaker 1: an added twist to an already difficult path. I think
Speaker 1: if Magically does survive, it's going to be a very
Speaker 1: different entity than the mysterious one we have seen, you know,
Speaker 1: talked about since two thousand twelve or so. I hope
Speaker 1: it sticks around. I want to see mixed reality become
Speaker 1: a more universal technology, but it's going to take a
Speaker 1: lot of time and development, and that cost has to
Speaker 1: come down, and that's going to that's a that's a
Speaker 1: that's a big challenge. I just don't know how you
Speaker 1: managed to do it. In the short term. It maybe
Speaker 1: years before we see anything that is truly compelling and
Speaker 1: practical and affordable. I hope we see it. We'll have
Speaker 1: to wait, alright, guys. That wraps up this episode of
Speaker 1: tech Stuff. If you have any suggestions for topics I
Speaker 1: should cover in future episodes, reach out to me on Twitter.
Speaker 1: The handle is tech stuff h s W and I'll
Speaker 1: talk to you again really soon. Text Stuff is an
Speaker 1: I Heart Radio production. For more podcasts from I Heart Radio,
Speaker 1: visit the I Heart Radio app, Apple Podcasts, or wherever
Speaker 1: you listen to your favorite shows h