Speaker 1: Ill conceived physics experiments, reckless experiments with viruses, unfriendly superintelligent AI.
Speaker 1: Dealing with each one of these will be a quagmire
Speaker 1: onto itself. But remember, existential risks are like nothing we've
Speaker 1: ever encountered before. We humans haven't been prepared by millennia
Speaker 1: of evolution like we have for other disasters. We're not
Speaker 1: equipped out of the box to deal with the existential
Speaker 1: risks that loom ahead in our near future. In fact,
Speaker 1: it's almost as if we're wired not to be able
Speaker 1: to deal with them properly. And the more we look
Speaker 1: into it, it turns out the question of whether we'll
Speaker 1: be able to navigate our existential risks to a safe
Speaker 1: future is actually the same question as whether we'll be
Speaker 1: able to overcome ourselves. But for the moment, let's leave
Speaker 1: all that and go inside your body instead. You have
Speaker 1: tiny invisible robots moving through your blood stream. I should
Speaker 1: say we're in the future. Let's say it's for the
Speaker 1: sake of keeping numbers nice and round, and pretty much
Speaker 1: everyone has tiny invisible robots moving through their blood stream.
Speaker 1: It's a good thing, actually, because these tiny robots act
Speaker 1: as a human design backup force for your immune system.
Speaker 1: You took them in a pill back in and the
Speaker 1: moment you pop that capsule into your mouth, your healthy
Speaker 1: life expectancy increased by a hundred years. As the enzymes
Speaker 1: in your gut began to dissolve the capsule, your digestive
Speaker 1: fluids poured into it, and the sudden change in temperature
Speaker 1: and pH activated the first generation of nanobots inside. They
Speaker 1: came online, connected to their shared WiFi network, activated their
Speaker 1: propulsion systems, and passed through your gut wall into your
Speaker 1: blood stream, fanning out through your body. Over the years,
Speaker 1: each of those first gen nanobots assemble copies of itself,
Speaker 1: and those copies make copies, and now three years on,
Speaker 1: you have a stable colony of tiny, invisible robots living
Speaker 1: inside of you. They search for pathogens to destroy. They
Speaker 1: prune cells that show signs of growing into tumors and
Speaker 1: repair the DNA inside to make sure they won't turn
Speaker 1: cancerous again. The clear plaque from the interiors of your
Speaker 1: blood vessel. They assist insulin in removing sugars, fats, and
Speaker 1: proteins from your blood stream after you eat for storage.
Speaker 1: Later on, they assist in clearing neurotransmitters from your synapses
Speaker 1: after you've had a thought. They target fats to burn
Speaker 1: in areas of your body that you select through their
Speaker 1: app Everything your body did before, or should have done
Speaker 1: to keep itself in harmony, it does remarkably better now
Speaker 1: since you took that capsule back in. The nanoscale is
Speaker 1: the scale of atoms. It's the smallest scale that we're
Speaker 1: able to manipulate, and we've only recently become able to
Speaker 1: do that. I should say we're back in the present
Speaker 1: time now. Depending on the species, a female mosquito will
Speaker 1: drink about five micro leaders of your blood five millions
Speaker 1: of a leader before flying off. Inside those five micro
Speaker 1: leaders slashing around to that mosquito's tiny stomach are around
Speaker 1: twenty five million red blood cells. Just one of those
Speaker 1: red blood cells is made up of around one and
Speaker 1: twenty trillion atoms, and just one single hydrogen atom is
Speaker 1: a tenth of a nanometer in size. That's the scale
Speaker 1: of the world where nanobots will dwell. On this tiny level,
Speaker 1: nanobots are expected to eventually be able to do amazing things,
Speaker 1: magical things in the Arthur C. Clark sense of the term.
Speaker 1: There are so many promises with nanotechnology that perhaps no
Speaker 1: other emerging field has such a wide scope of applications
Speaker 1: ready and waiting to be applied. Because they are the
Speaker 1: size of atoms, nanobots will be able to rearrange atoms,
Speaker 1: and so the materials they make will be manufactured to
Speaker 1: atomic precision. To us up here on the human scale,
Speaker 1: the things nanobots make will be flawless, since virtually any
Speaker 1: material could be turned into any other material. Anything will
Speaker 1: qualify as raw material for anything else, which means that
Speaker 1: our current global waste problem will vanish, a happy byproduct
Speaker 1: of the global increase in material wealth that nanobots will provide.
Speaker 1: This will also mean the end of scarcity, since anyone
Speaker 1: with the nano factory at home, which will eventually be
Speaker 1: everyone as the technology spreads, will be able to make
Speaker 1: whatever they like. But despite all of the golden promises,
Speaker 1: nanotechnology potentially holds in store, as we saw in the
Speaker 1: chapter on artificial intelligence, it poses an existential threat to
Speaker 1: us as well. Like every other technology that poses an
Speaker 1: existential threat, it is dual use. It can be used
Speaker 1: to create both positive and negative outcomes for us, and
Speaker 1: really you can make the same case for basically any
Speaker 1: technology we humans have ever come up with Just to
Speaker 1: take one example, you can use paper towels as a
Speaker 1: handy way to clean up a spill or to start
Speaker 1: a house fire. But as with all other existential risks,
Speaker 1: the potential negative outcomes associated with nanotechnology have a vastly
Speaker 1: wider scope than a house fire started with the roll
Speaker 1: of paper towels. There is, of course, that unpleasant outcome
Speaker 1: where in the way of our arch enemy, the paper
Speaker 1: clip maximizer, they disassemble us for use in some other form.
Speaker 1: But even now, in the air before nanobots, we've already
Speaker 1: identified hazards from the current nanotechnology we have today. Because
Speaker 1: of their minute size, nanoparticles can irritate the lung tissue
Speaker 1: of humans who breathe them in much the same way
Speaker 1: that asbestos and silica can, possibly leading to cancer if
Speaker 1: the scar tissue that results isn't repaired in the body properly.
Speaker 1: Today's nanoparticles are also concerning because they are inorganic and
Speaker 1: there's no mechanism for them to degrade, which means they
Speaker 1: may persist in the environment forever as far as we
Speaker 1: can tell right now. Ironically, both of these current problems
Speaker 1: with nanoparticles, they're potentially cause cancer and the possibility they
Speaker 1: persist forever can be solved by the nanobots of the future.
Speaker 1: There are other speculative ways that nanotechnology could turn out poorly.
Speaker 1: The most famous of the mall it was called the
Speaker 1: gray goo hypothesis, which was first put into words back
Speaker 1: in x by m I T Engineering professor and Future
Speaker 1: of Humanity Institute member Eric Drexler in his book Engines
Speaker 1: of Creation. Gray Goo, as Drexler's pointed out, is a
Speaker 1: possible outcome from a poorly considered nanotech design where nanobots
Speaker 1: capable of replicating themselves and able to sustain themselves using
Speaker 1: energy harvested from the environment, say from plant material, would
Speaker 1: be able to exist outside of our control. At a
Speaker 1: certain point, they might enter a runaway exponential population explosion
Speaker 1: where their numbers grow so massive that they collectively become
Speaker 1: visible to us. Is what would seem like a fluid,
Speaker 1: gooey substance actually made up of untold numbers of nanobots,
Speaker 1: all feeding on our environment, eventually overwhelming Earth and ruining
Speaker 1: the global biosphere, not to mention resulting in the eventual
Speaker 1: extinction of humanity. Ye Drexler has since publicly denounced his
Speaker 1: gray goog hypothesis, pointing out that it could only arise
Speaker 1: from an obvious and foreseeable flaw and design, not some
Speaker 1: sort of trait that's inherent in nanobox. Drexler believes his
Speaker 1: hypothesis planted a seed in the media, which grew into
Speaker 1: a sensational thicket of vines covering the real work of
Speaker 1: nanotechnologists and choking the life from the field of research
Speaker 1: that he helped establish. A future event like gray goo
Speaker 1: eating the world from around us would qualify as what
Speaker 1: existential risk. Philosopher Nick Bostrom calls subsequent ruination. Things start
Speaker 1: out for us just fine with the technology that we've
Speaker 1: built and mastered, but it somehow takes an unexpected left turn,
Speaker 1: and things ultimately end up broughten for us because of it,
Speaker 1: resulting in our eventual extinction. Bostrom wasn't the first philosopher
Speaker 1: to think about existential risks, but he innovated how we
Speaker 1: see them. Here's Bostrom's colleague, philosopher Toby Ord. Thinking about
Speaker 1: existential risks really started in the twentieth century with the
Speaker 1: advent of nuclear weapons and the threat of major nuclear war.
Speaker 1: In the nineteen sixties, Bertrand Russell wrote about the threat
Speaker 1: of human extinction due to nuclear weapons, and in the
Speaker 1: nineteen eighties there are a few people, almost simultaneously who
Speaker 1: really got the bigger picture about extinction. Those people were
Speaker 1: Jonathan Shall, Carl Sagan, and Derek Parfit. Then in the
Speaker 1: nineteen nineties, John Leslie wrote a fantastic book on extinction
Speaker 1: called The End of the World Um. And then in
Speaker 1: the two thousands, Nick Bostrom, my colleague, he really made
Speaker 1: a large number of major breakthroughs on this area. He
Speaker 1: was the one who expanded it out from ext action
Speaker 1: to existential risk, including a large number of other possibilities.
Speaker 1: All of what they will have in common is that
Speaker 1: they would be the permanent loss of humanity's potential. Nick
Speaker 1: Bostrom realized that there are other possible outcomes of existential
Speaker 1: catastrophes beyond just the extinction of our species. There are,
Speaker 1: he realized, some fates for humanity that are even worse
Speaker 1: than death. There is of course, subsequent ruination like grey Goo,
Speaker 1: where our technology takes a bad turn and extinction follows later.
Speaker 1: But Bostrom also realized that we humans don't actually need
Speaker 1: to go extinct to undergo an existential catastrophe. There are
Speaker 1: some scenarios where we could be broken as a species,
Speaker 1: left to limp along and definitely without the possibility of
Speaker 1: ever regaining the place in history where we fell from.
Speaker 1: Perhaps a virus killed most of humanity, and the genetic
Speaker 1: bottleneck that resulted lead to humans who were no longer
Speaker 1: capable of solving extremely complex problems, or who lost the
Speaker 1: ability to coordinate with one another in large groups. Our
Speaker 1: species would be alive, sure, but our existence would be
Speaker 1: a shadow of what it was before, and the potential
Speaker 1: sunny future that may have been in store for humanity
Speaker 1: would be lost forever. Eventually, this loss of potential would
Speaker 1: be made permanent when a natural existential risk like an
Speaker 1: asteroid or supervolcano came along tens or hundreds of thousands
Speaker 1: of years down the road and drove us to extinction
Speaker 1: once and for all. Bostrom calls this kind of scenario
Speaker 1: permanent stagnation, where the existential catastrophe comes long before extinction.
Speaker 1: It's a kind of catastrophe from which we could not
Speaker 1: recover possible end to the human story. Some way to
Speaker 1: permanently lock ourselves into some radically suboptimal state. There's also
Speaker 1: flawed realization an algorithm we create growing super intelligent beyond
Speaker 1: our expectations, and running a muck is an example of that.
Speaker 1: It's akin to subsequent ruination, but without giving us even
Speaker 1: the brief period where we get to enjoy the full
Speaker 1: benefits of the technology before things go badly for us
Speaker 1: because of it. As much as Eric Drexler wishes that
Speaker 1: he had never written the words gray goo, he may
Speaker 1: have very well saved the world when he did, for
Speaker 1: better or worse. For the people currently working in the
Speaker 1: field of nanotechnology, he identified a potential catastrophic outcome that
Speaker 1: we can plan to design against. One of the great
Speaker 1: benefits of thinking about the existential risk nanotechnology poses is
Speaker 1: that it's still in its infancy as a field and
Speaker 1: can be guided in safe ways, so that when we
Speaker 1: do live in a world of nanobots, we can be
Speaker 1: assured that they won't pose a threat then or down
Speaker 1: the road. But how we get there? How do we
Speaker 1: plan for a technology that doesn't actually exist yet in
Speaker 1: a field of research that only a minute fraction of
Speaker 1: humans actually understand and feel qualified to talk about. How
Speaker 1: do we manage the media's understanding of the issues surrounding
Speaker 1: the field so that it doesn't cause unjustified panic among
Speaker 1: the public, which could turn against it and choke the
Speaker 1: life from it once and for all. And just as important,
Speaker 1: how do we ensure that the corporate and academic labs
Speaker 1: working on nanotechnology don't pursue dangerous lines of research and design.
Speaker 1: If you've been asking yourself questions like these about the
Speaker 1: other existential risks I've talked about so far in this series,
Speaker 1: then you may have already hit upon the idea that
Speaker 1: we might need a singleton to guide us through the
Speaker 1: coming years to technological maturity. A singleton, in the sense
Speaker 1: that Nick Bostrom has applied it to existential risks, is
Speaker 1: a body capable of making the final decision for everyone
Speaker 1: on the plan in it. I'll let him explain it
Speaker 1: has singled on. Is uh just a world order? We're
Speaker 1: at the highest level of decision making. There is only
Speaker 1: one decision making process. So, in other words, our world
Speaker 1: is where global coordination problems are. At least the most
Speaker 1: varitable coordination problems have been solved, So no more wars
Speaker 1: or arms races or technology races. Our pollution and destruction
Speaker 1: of the global commons one of the biggest challenges we
Speaker 1: will face in the coming decades and centuries. Is coordinating
Speaker 1: on a global level, In other words, getting everyone to
Speaker 1: agree on the best way to move forward and addressing
Speaker 1: existential risks. We will need to study the issues, funnel
Speaker 1: some of the world's brightest minds towards identifying future existential risks,
Speaker 1: throw lots and lots of money at the problems, and
Speaker 1: figure out the best, safest way forward towards technological maturity.
Speaker 1: But all the study, bright ideas, intricately mapped ways forward
Speaker 1: don't amount to anything if one person can undermine everything
Speaker 1: with a single accident. So we will need every single
Speaker 1: country on Earth to buy into this process right now.
Speaker 1: The geopolitical arrangement on Earth is based on the sovereignty
Speaker 1: of nations. Each country has its own borders and citizenry,
Speaker 1: and it's up to the country's government to make its
Speaker 1: own decisions. There are lots of exceptions to this. Some
Speaker 1: governments make agreements that stitch their nations together to some degree,
Speaker 1: as seen in the European Union or the North American
Speaker 1: Free Trade Agreement, and sometimes one nation will invade another nation,
Speaker 1: resorting to force to influence the other government's decisions. For
Speaker 1: the most part, though the nations of the world leave
Speaker 1: it to the other nations of the world to make
Speaker 1: their own choices about how they function, this won't really
Speaker 1: work in tackling a essential risks. We will need to
Speaker 1: all agree to abide by whatever we decide is the
Speaker 1: best way to proceed. But getting to this level of
Speaker 1: consensus can be messy, and you could see just how
Speaker 1: it could get that way with existential risks by taking
Speaker 1: a look at how humans have dealt with climate change.
Speaker 1: The Inner Governmental Panel on Climate Change the i p
Speaker 1: c C is an offshoot of the United Nations that
Speaker 1: was set up back to study climate change and provide
Speaker 1: the world's governments with the best science about the issue
Speaker 1: and how to tackle it. An issue like climate change
Speaker 1: requires international cooperation because climate change effects everyone. It crosses
Speaker 1: the borders of the world, and so not only does
Speaker 1: it affect everyone, it also requires action from everyone. To
Speaker 1: combat climate change, we need the cooperation of all nations
Speaker 1: for the collective public good, and it's exactly the same
Speaker 1: with existential risks. The i p c C was chartered
Speaker 1: during a time when global geopolitics respects the sovereignty of nations,
Speaker 1: and that has proven a problem for it to take
Speaker 1: one example, Back in two thousand seven, when the i
Speaker 1: p c C issued its fourth Assessment on Global Climate Change,
Speaker 1: the body's reports on the current cutting edge scientific understanding
Speaker 1: of the issue. Words spread to media that the report
Speaker 1: had been watered down by diplomacy. Saudi Arabia, one of
Speaker 1: the world's leading producers of fossil fuels, and China in
Speaker 1: the United States, two of the world's leading producers of
Speaker 1: emissions from burning those fossil fuels, use their influence to
Speaker 1: temper the report's findings on how fossil fuel use contributes
Speaker 1: to climate change, to make fossil fuel's role seem less
Speaker 1: scientifically certain. As a result, the public was presented with
Speaker 1: findings that seemed much more doubtful about the role of
Speaker 1: fossil fuel emissions and climate change, a out that's still
Speaker 1: alive today. This could not be allowed to happen with
Speaker 1: existential risks. Climate change is one of the most important
Speaker 1: issues facing humanity today. Existential risks are the most important.
Speaker 1: So how do we create a body that's immune to
Speaker 1: diplomatic and economic pressures of countries as strong as the
Speaker 1: U S, Saudi Arabia, and China. Answer is a singleton
Speaker 1: Our hypothetical singleton could arise from an international body organized
Speaker 1: to study and deal with existential risks. Let's call it
Speaker 1: our Existential Risks Commission. Just out of necessity, as the
Speaker 1: world wakes up to the real scope and severity of
Speaker 1: these risks, we may give that commission an enormous amount
Speaker 1: of power to override any nation's opposition to its findings
Speaker 1: and guidelines, which would mean an enormous change for global geopolitics,
Speaker 1: but one we would likely feel was nest necessary. Our
Speaker 1: Existential Risks Commission would need to have teeth. One way
Speaker 1: it might ensure compliance among all nations is through a
Speaker 1: global surveillance network. We would need to keep tabs on
Speaker 1: all the scientists who work in fields that pose an
Speaker 1: existential threat, to make sure that they weren't secretly working
Speaker 1: on designs or experiments the Commission deemed too risky to pursue.
Speaker 1: The same goes for corporations that make products that use
Speaker 1: risky technology. Our Commission would need to keep tabs on
Speaker 1: everyone really to monitor for signs of a black market
Speaker 1: developing and banned technology. So each government would be required
Speaker 1: to set up a surveillance network within its own borders
Speaker 1: and the existential risks, Commission would have access and ultimate
Speaker 1: control over all of them. It should probably also monitor
Speaker 1: each nation's government as well, and we would probably also
Speaker 1: need to grant our Commission with some sort of military
Speaker 1: or policing power as a last resort, with a force
Speaker 1: that is capable of overwhelming any nations in the world.
Speaker 1: Or perhaps to make it easier, we would just allow
Speaker 1: the Commission to disband the world's militaries and maintain its
Speaker 1: own small force it could use to invade and easily
Speaker 1: occupy any non complying nation. With a single decision making
Speaker 1: body in charge of determining the best way forward towards
Speaker 1: technological maturity, one equipped with unchecked authority, able to monitor
Speaker 1: every person alive on the planet, and to use the
Speaker 1: threat of violence to ensure that we all stay in
Speaker 1: line on our march toward a safe future. We may
Speaker 1: just make it through the next century or two and
Speaker 1: arrive at a point where the future of humanity is assured.
Speaker 1: But as you may have noticed as I was describing it,
Speaker 1: a singleton can also pose an existential threat itself. That
Speaker 1: same global body we create to manage our existential risks
Speaker 1: could easily become totalitarian forming a permanent global dictatorship that
Speaker 1: no future generation could possibly overthrow m And it's about
Speaker 1: here that you might start to feel like, no matter
Speaker 1: what we do, humanity is doomed. In the early nineteen seventies,
Speaker 1: the world started thinking about the environment. Everything we think
Speaker 1: of as normal today, recycling not throwing your trash out
Speaker 1: of your car window, using less energy, generally considering ourselves
Speaker 1: as stewards of the global biosphere. All of that finds
Speaker 1: its origin in the early seventies, and it's largely because
Speaker 1: of two books that came out around then. In Stanford University,
Speaker 1: entomology professor Paul Arelick and his wife Anne published a
Speaker 1: book they co wrote called The Population Bomb. It was
Speaker 1: the culmination of years of Airlic's thoughts about the sustainability
Speaker 1: of the massive increase in population of humans and our
Speaker 1: effects on the Earth's finite resources. He decided that the
Speaker 1: outlook was not good. Aerlic prophesies that by the middle
Speaker 1: of the seventies the world would begin to see massive
Speaker 1: die offs of humans from starvation as we surpassed agricultures
Speaker 1: carrying capacity. People didn't pay attention to the Airlocks books
Speaker 1: until Dr Erlic appeared on Late Night with Johnny Carson
Speaker 1: in nineteen seventy and spoke about the coming horror for
Speaker 1: an hour. Then they really began to pay attention. Around
Speaker 1: the time Paul Erlic was on Late Night, a handful
Speaker 1: of scientists from around the world have been assembled into
Speaker 1: a group by a wealthy Italian industrialist. They were called
Speaker 1: the Club of Rome. The scientists had devised computer models
Speaker 1: to build forecasts of humanity's future based on trends like
Speaker 1: resource use, pollution, and population growth. They saw pretty much
Speaker 1: the same doom in their crystal ball that Aerlic did,
Speaker 1: mass starvation, collapsing society, widespread pollution, and the attendant negative
Speaker 1: impacts on health that carries. The only silver lining to
Speaker 1: the Club of Rome's report, which they called The Limits
Speaker 1: to Growth, was that we had perhaps until before we
Speaker 1: saw the worst of it. Both books and the media's
Speaker 1: coverage of them got the world's attention, but this was
Speaker 1: not a new idea. The Club of Rome and Paul
Speaker 1: Alick followed in the tradition of Thomas Malthus, the eighteenth
Speaker 1: century clergyman and demographer who was the first to write
Speaker 1: about the limits of agriculture. Malthus pointed out that while
Speaker 1: humans can multiply exponentially the resources we get from the Earth,
Speaker 1: what we call natural capital, do not, which means that
Speaker 1: because of our propensity to place an emphasis on growing
Speaker 1: our species, we humans are essentially doomed to outstrip Earth's
Speaker 1: resources at some point, including, as Malthus pointed out, our
Speaker 1: food supply. In the mid sixties, before Airlick's book was published,
Speaker 1: there was widespread famine in India, and during the seventies
Speaker 1: and eighties there were additional widespread payments in the Horn
Speaker 1: of Africa. But if anything, the Population Bomb is the
Speaker 1: story of a global catastrophe that was averted. As bad
Speaker 1: as the famines around the world have been, things could
Speaker 1: have been much much worse. Unbeknownst to most of the world.
Speaker 1: Thirty years before The Population Bomb and The Limits to
Speaker 1: Growth were published, a few groups, like the Rockefeller Foundation
Speaker 1: began working to figure out how to expand the carrying
Speaker 1: capacity of agriculture, and they were successful, thanks in large
Speaker 1: part to a man raised on a farm in Iowa
Speaker 1: named Norman Borlog. Borlog had been hired by the Rockefeller
Speaker 1: Foundation to oversee their research station in Mexico, which was
Speaker 1: established with the Mexican government to find ways to improve wheat.
Speaker 1: It's difficult to think of any kind of work that
Speaker 1: sounds more boring than improving wheat, but Borlog managed to
Speaker 1: do just that. He improved wheat, and today he is
Speaker 1: widely and frequently credited with saving the lives of a
Speaker 1: billion people who would have otherwise starved to death without it.
Speaker 1: Brologs high yield wheat and an improved type of rice
Speaker 1: that was developed at the same time at another research
Speaker 1: station in the Philippines could triple the amount of grain
Speaker 1: a single plant could produce, which means that farmers could
Speaker 1: suddenly get three times more grain from the same amount
Speaker 1: of land. So both the food supply and the income
Speaker 1: of much of the world's global poor increased dramatically in
Speaker 1: a very short time, which means that the world was saved.
Speaker 1: Between nineteen seventy and nineteen seventy five, the amount of
Speaker 1: rice produced in Asia grew by on it doubled in
Speaker 1: just a few years. The needle on Earth's caring capacity
Speaker 1: for agriculture moved from quivering worryingly at the red line
Speaker 1: along the end of the dial to somewhere comfortably back
Speaker 1: around the halfway mark. Norman Borlog rightly or in the
Speaker 1: nineteen seventy Nobel Peace Prize for his work Ever the
Speaker 1: White Knight. He used the attention to stress that he
Speaker 1: only bought the world some breathing room while it figured
Speaker 1: out how to deal with its population monster, as he
Speaker 1: called it. The story of how Norman Borlog defused Paul
Speaker 1: Alick's population bomb pretty well gets across the idea of
Speaker 1: what's called techno optimism. Techno optimism is the full faith
Speaker 1: some people play some technology to get us out of
Speaker 1: any jam. Really, it's faith in human ingenuity. One proposal
Speaker 1: for dealing with global warming brought on by climate change
Speaker 1: is to add air assaults that reflects solar radiation into
Speaker 1: the atmosphere. We would in effect be bolstering the atmosphere's
Speaker 1: ability to already do this, lending a very important natural
Speaker 1: process a technological hand. This would actually be a comparatively
Speaker 1: easy task. We could do it with current technology, and
Speaker 1: so a techno optimist would say, we probably don't need
Speaker 1: to worry much about global temperatures from climate change, since
Speaker 1: we already have a way of inventing a solution. But
Speaker 1: what if a hundred years from now we find that
Speaker 1: those aerosols we've added are working too well. Global temperatures
Speaker 1: are actually starting to drop, and our crops are in
Speaker 1: danger of failing worldwide. No problem. A hundred years from now,
Speaker 1: we will almost certainly have mastered nanotechnology. We can just
Speaker 1: deploy them into the atmosphere to deal with the issue.
Speaker 1: We could probably program them so they not only disintegrate
Speaker 1: the aerosols, they could also rearrange them into a different
Speaker 1: type of aerosol, like black soot or sulfates that absorb sunlight,
Speaker 1: which would heat the glow back up more quickly, so
Speaker 1: we could avoid those widespread crop failures back on Earth.
Speaker 1: In fact, now that we think of it, we might
Speaker 1: as well just leave the nanobots up there to keep
Speaker 1: tabs on global tempera tures and adjust the atmosphere is
Speaker 1: ability to absorb or reflect solar radiation at any given moment,
Speaker 1: kind of like how they'll eventually keep our bodies humming
Speaker 1: along in an optimal state. But what if our nanobots
Speaker 1: turn out to not have been designed perfectly and they
Speaker 1: end up entering a runaway replication scenario like the gray
Speaker 1: goo hypothesis or something worse. At this point, most techno
Speaker 1: optimists would pinch the bridge of their nose and try
Speaker 1: to muster more patients gray goo, they might answer, is
Speaker 1: almost certainly not going to happen, and even if it did,
Speaker 1: it would be so far off in the future that
Speaker 1: you can rest assured we would find a way to
Speaker 1: use some other type of technology we haven't even thought
Speaker 1: of yet to handle it. The more you drill into
Speaker 1: any given problem, the more it seems like technology can
Speaker 1: get us out of it. It has so far, and
Speaker 1: it probably can continue to in the future. But the
Speaker 1: fatal flaw of techno optimism is that it tends to
Speaker 1: discourage planning foresight, which I'm hoping that by now in
Speaker 1: the series you've come to realize is of vital importance.
Speaker 1: Rather than taking steps to head off the problem today,
Speaker 1: like reducing carbon emissions, we can instead keep a fairly
Speaker 1: sunny outlook that we will eventually handle it with aerosols,
Speaker 1: then nanobots, then something else we haven't even thought of yet.
Speaker 1: The problem, though, is that if any link in that
Speaker 1: chain breaks down, if the innovation doesn't work or it
Speaker 1: comes too late, then we've missed our chance to avoid
Speaker 1: the crisis. And there's another issue with techno optimism. Sometimes
Speaker 1: our solutions to the problem actually make things worse. You
Speaker 1: would be hard pressed to find a person alive that
Speaker 1: faulted Norman Borlog for his work. But the green revolution,
Speaker 1: the expansion of agriculture's care and capacity that he midwifed,
Speaker 1: requires farmers to use enormous amounts of fertilizer and irrigation,
Speaker 1: which tends to cause runoff to waterways that absorb the
Speaker 1: nutrients themselves, harming the aquatic ecosystems. This intensive farming also
Speaker 1: depletes the nutrients in the soil, which means that today,
Speaker 1: decades after Borlogs wheat made its debut, farmers put in
Speaker 1: more fertilizer than ever before, while the amount of food
Speaker 1: they harvest has plateaued. To put the icing on the cake,
Speaker 1: the fertilizer requires large inputs of energy to produce, which
Speaker 1: in two thousand and eleven amounted to emissions of about
Speaker 1: six billion metric tons of greenhouse gases from the world's
Speaker 1: farm were about of the global total. Rather than taking
Speaker 1: the breathing room Borlog gave us to deal with the
Speaker 1: underlying issues we have. The collective techno optimism it brought
Speaker 1: on encouraged us to just kick the can down the road.
Speaker 1: Well we've come upon the can again. Hopefully we'll figure
Speaker 1: out a way to kick it. Those unwanted knock on
Speaker 1: effects of the green revolution of techno optimism, even the
Speaker 1: hypothetical singleton I talked about earlier, they formed the basis
Speaker 1: of an argument against taking steps to mating existential risks.
Speaker 1: Doing something could possibly make things even worse. You could
Speaker 1: call this the Gilligan effect. Helpfulness results in calamity. It is,
Speaker 1: perhaps unsurprisingly, not the only argument people make against taking
Speaker 1: existential risks we face seriously. It is to our great
Speaker 1: misfortune that we are being presented with the responsibility of
Speaker 1: dealing with existential risks at this point in human history.
Speaker 1: It was only perhaps fifty to a hundred thousand years
Speaker 1: ago that humans started being born with the full package
Speaker 1: of behaviors and intelligence that make us uniquely human. Our
Speaker 1: ability to reason and think abstractly, to imagine different futures,
Speaker 1: our ability to organize. Imagine if we had had another
Speaker 1: hundred thousand years to continue to evolve before the existential
Speaker 1: risks will have to address appeared on our horizon. But
Speaker 1: that's not how the chips have fallen. Instead, it has
Speaker 1: come upon us while we are in what Carl Sagan
Speaker 1: called our technological adolescence the most dangerous phase on the
Speaker 1: way to technological maturity. It is up to those of
Speaker 1: us alive in the twenty one century. We bear responsibility
Speaker 1: for saving the future of the human race, and we
Speaker 1: have come up with plenty of reasons why we shouldn't, or,
Speaker 1: more to the point, why we won't. Probably first among
Speaker 1: them is that the chance of one of these risks
Speaker 1: befalling us is so small, so utterly remote, they're not
Speaker 1: even worth considering. It is true that the chance of
Speaker 1: an existential catastrophe like an altered pathogen escaping a lab
Speaker 1: and creating a pandemic, is extremely remote, But as more
Speaker 1: labs conduct more risk experiments around the world, the probability
Speaker 1: of that remote risk begins to compound. And the same
Speaker 1: is true in other fields. As new particle colliders run
Speaker 1: higher energy experiments, as more companies deploying more self improving
Speaker 1: algorithms on the global networks, what was once a mere
Speaker 1: remote possibility of existential catastrophe becomes decidedly less remote. I
Speaker 1: think there are various forms of arguments against dealing with existentialies,
Speaker 1: some of them much better than others. So I think
Speaker 1: the heads in the sound ob the action, Oh, it
Speaker 1: won't happen or it hasn't ever happened before. That is
Speaker 1: a really bad one, But it's of course psychologically rabbi
Speaker 1: common because it fits with the cognitive biases. That was
Speaker 1: Ander Sandberg, philosopher from the Future of Humanity Institute. In
Speaker 1: addition to all of the advanced behaviors that we humans
Speaker 1: have evolved that had served us so well, we also
Speaker 1: operate using some extremely ancient techniques too, short cuts that
Speaker 1: allow us to deal with everyday life, but can break
Speaker 1: down when we're faced with things that are out of
Speaker 1: the ordinary. What results are called biases. Take, for example,
Speaker 1: being presented with the fork in the road, given that
Speaker 1: both paths look equally inviting, we might have trouble choosing.
Speaker 1: But say we've been presented with the same decision elsewhere,
Speaker 1: with other forks and other roads before, and we've usually
Speaker 1: taken the left path. Since nothing bad happened to us
Speaker 1: all those other times we've chosen to go left, we
Speaker 1: would feel pretty sure nothing will this time either, So
Speaker 1: we head down the left path this time too, whistling
Speaker 1: without a care in the world, totally unaware of the
Speaker 1: family of hungry bears ahead. Our cognitive biases can make
Speaker 1: us overconfident, suspicious of new things, optimistic, pessimistic, frozen within decision.
Speaker 1: We are, you could say, a little hamstrung by them.
Speaker 1: But even when we managed overcome our biases or set
Speaker 1: them aside, which we will need to when we're dealing
Speaker 1: with existential risks, there are still plenty of other reasons
Speaker 1: we can come up with to avoid addressing them. For one,
Speaker 1: even discussing this type of risk can be dangerous. Such
Speaker 1: talk can have a chilling effect on a field that's
Speaker 1: struggling to establish itself, as Eric Drexler found when he
Speaker 1: let the gray Goo genie out of the bottle and
Speaker 1: engines of creation, talking about things like AI becoming super
Speaker 1: intelligent and taking control of our world can really go
Speaker 1: a long way to turning the public off from the
Speaker 1: idea of scientists working on building self improving thinking machines. Besides,
Speaker 1: as most machine intelligence researchers will point out, at this
Speaker 1: stage in its development, the field is capable of producing
Speaker 1: a machine that's perhaps as smart as a three year old,
Speaker 1: or even if it is advanced, it's advanced at just
Speaker 1: one thing like finding patterns and medical charts or identifying
Speaker 1: cap pictures, we don't need to worry about a. In
Speaker 1: other words, this argument seems shortsighted. If it is the
Speaker 1: case that we're at a point where we can still
Speaker 1: fully control our artificial intelligence, then now is the best
Speaker 1: time to plan for the potential future outcomes they might bring,
Speaker 1: so that we can ensure as best as we can
Speaker 1: that they will continue to remain under our control. It's
Speaker 1: probably not the best idea to wait until tomorrow simply
Speaker 1: because they don't pose a threat today. Here is Oxford
Speaker 1: philosopher Sebastian Farquhar. If we had started working on really
Speaker 1: small nuclear explosives in the eighties and thought that we
Speaker 1: could sort of maybe c OA to scale it up
Speaker 1: to weapons with vastly more explosive power than we'd ever
Speaker 1: imagined before, but it wasn't quite clear that it was
Speaker 1: going to work or not. Um, I think it would
Speaker 1: have been irresponsible at that point not to invest at
Speaker 1: least some thought and what would happen if nuclear weapons
Speaker 1: did reach the stage that they reached the forties. And
Speaker 1: so that's sort of where I see us now, is not,
Speaker 1: you know, not confidently saying super intelligent a g I
Speaker 1: is around the corner, but rather saying, you know, this
Speaker 1: might be it might be a turning point for um
Speaker 1: intelligence on this planet, and if that turning point is
Speaker 1: around the corner, it would be useful for some people
Speaker 1: to start laying the groundwork for making that turning point safe.
Speaker 1: But it's difficult to fault people who work in fields
Speaker 1: like AI and others. The very same people who witness
Speaker 1: firsthand the extremely slow and frustrating progress in the state
Speaker 1: of the technology that we on the outside don't see,
Speaker 1: not to mention it's their careers that are on the
Speaker 1: line off the public turns cold to their field. People
Speaker 1: who work in AI, nanotechnology, particle physics, and other fields
Speaker 1: that will eventually emerge have dedicated and will dedicate their
Speaker 1: adult lives to this research. Currently, we rely on these
Speaker 1: same people who are working on the science and technology
Speaker 1: that may pose an existential risk to tell us whether
Speaker 1: they're safe or not, which puts the whole world in
Speaker 1: a very difficult position. Here's Eric Johnson, the law professor
Speaker 1: who investigated the potential risks posed by particle colliders. I
Speaker 1: don't think there's any particle physicists out there who are
Speaker 1: mad scientists bent on destroying the earth. They're they're good people,
Speaker 1: and there's there's none of them who are you know,
Speaker 1: as sociopaths who would uh knowingly put the earth at
Speaker 1: a at a big risk of being destroyed. But there's
Speaker 1: a question about when you're when you're making these subjective
Speaker 1: judgments about how to build this model. If you are
Speaker 1: self interested, if if your employer, if all of your friends,
Speaker 1: if your whole professional life is built around this community,
Speaker 1: this project, are you likely to go a little easier
Speaker 1: on the risk assessment than someone else might be. And
Speaker 1: I think that that's an open question. I think it's
Speaker 1: a fair question. In addition to careers, there's also money
Speaker 1: at stake, not just public funding for research projects at universities,
Speaker 1: but perhaps most intractable of all corporate profits. Scaring the
Speaker 1: public can cause these funds and these profits to dry up,
Speaker 1: which has a real world effect like people losing their
Speaker 1: jobs and fields of research freezing over. It's happened before.
Speaker 1: In October, the US Congress effectively shuttered the American physics
Speaker 1: community when it cut off funding for the super Conducting
Speaker 1: super Collider, a particle accelerator outside of Dallas. It would
Speaker 1: have had a track almost four times the circumference of
Speaker 1: the Large Hadron Collider and would have been capable of
Speaker 1: achieving particle collisions at three times the energy of the LHC.
Speaker 1: It would have been a landmark particle collider, one with
Speaker 1: power that we still haven't reached today would have been
Speaker 1: but it never had the chance to because Congress decided
Speaker 1: that the project was too expensive and too difficult to understand.
Speaker 1: So after having spent already two billion dollars on the project,
Speaker 1: Congress withdrew any further support, and the super Conducting super
Speaker 1: Collider was never finished. Uh six months ago, the Congress
Speaker 1: voted to terminate the super Conducting super Collider project. As
Speaker 1: Eric Johnson explained in the previous chapter, the particle physics
Speaker 1: community tends to be arranged around the most powerful collider
Speaker 1: at any given time. So American physicists reeled from the
Speaker 1: loss of their collider and the subsequent funding cuts that
Speaker 1: followed in physics departments and universities across the country. In
Speaker 1: the meantime, the LHC began to rise, and the seat
Speaker 1: of physics moved through the Earth like a new trino,
Speaker 1: from beneath the plains of Texas to a hundred meters
Speaker 1: below the countryside between Switzerland and France. This underscores an
Speaker 1: extremely important point. The ways to alleviate existential risks in
Speaker 1: the future is to deal with them now. But dealing
Speaker 1: with them may mean that those of us living today
Speaker 1: would be asked to sacrifice our jobs or careers, money, comfort, health,
Speaker 1: all for the sole benefit of people who we will
Speaker 1: never meet, people whose great great grandparents great grandparents haven't
Speaker 1: even been born yet. To put it in other terms,
Speaker 1: what have future humans ever done for us? You could
Speaker 1: actually make a pretty good case that if they were
Speaker 1: able to figure out a way to pay us to
Speaker 1: alter our behavior so that their safe future was guaranteed,
Speaker 1: future humans would almost certainly give us whatever we wanted.
Speaker 1: But they can't, So it's entirely up to our good
Speaker 1: will to choose whether we will take steps to mitigate
Speaker 1: the threats to the future of the human rights, which
Speaker 1: doesn't necessarily bode well for the future of the human rights.
Speaker 1: Like we talked about before with climate change, there are
Speaker 1: things that everyone around the world shares resources, air, water,
Speaker 1: anything that everyone is affected by and benefits from. We
Speaker 1: call those things global commons. There's a widely held viewpoint
Speaker 1: that commons of any type must be collectively managed because
Speaker 1: we humans have a propensity to take as much as
Speaker 1: we can from them. If everyone has equal access to
Speaker 1: the commons, and the commons is some limited resource, then,
Speaker 1: speaking at a very basic level, every rational person has
Speaker 1: an incentive to take as much as they can before
Speaker 1: everyone else does. If this mentality is present among enough people,
Speaker 1: then we quickly deplete whatever common resources plentiful before. This
Speaker 1: is what ecologists Garrett Harden called the tragedy of the
Speaker 1: commons in a paper which he wrote among the same
Speaker 1: climate of doom that the population bomb was published in.
Speaker 1: It too follows the reasoning of Thomas Malthus. The commons
Speaker 1: works just fine until there are too many people taking
Speaker 1: too much from it. Then it crosses a threshold and
Speaker 1: it becomes spoiled for everyone. Harden wrote that he used
Speaker 1: the word tragedy not in the sense of unhappiness, but
Speaker 1: rather the remorseless working of things. The tragedy of the
Speaker 1: commons is in inevitability, he reasoned. Now it is true
Speaker 1: that at any moment, there are plenty of people who
Speaker 1: will take no more than their fair share from the commons,
Speaker 1: and in some cases even less, and some will act
Speaker 1: as stewards for the greater good. But the tragedy of
Speaker 1: the commons does exist, and we see it in resistance
Speaker 1: to things like caps on carbon dioxide emissions, which affect
Speaker 1: the global commons of the atmosphere. We have a hard
Speaker 1: enough time managing our current commons, but the future that
Speaker 1: we're being asked to protect is also a commons as well,
Speaker 1: and one with an added twist. Not only is the
Speaker 1: future of commons for those of us alive today, we
Speaker 1: also share it with those to come. If you think
Speaker 1: about what existential risk mitigation is, it's a common squared
Speaker 1: as it were, in that not only is it a
Speaker 1: global public good existential risk mitigation, but it's also of
Speaker 1: transgenerational public good in that most of the benefits would
Speaker 1: be bestowed to these titure generations that could come into existence.
Speaker 1: They have no say whatsoever and what we are doing now. Unfortunately,
Speaker 1: those people that come can't do anything to be good
Speaker 1: stewards of our shared comments. It's entirely up to us
Speaker 1: alive today to take whatever steps are necessary to protect
Speaker 1: the global pan generational commons that is the future, which
Speaker 1: makes people who haven't been born yet what economists call
Speaker 1: free riders. They reap the benefits of the sacrifices others
Speaker 1: make without contributing their fair share in this case, simply
Speaker 1: because it's impossible for them too. But we humans tend
Speaker 1: to resent all free riders, and not just us humans.
Speaker 1: We found behavior all over the animal kingdom that punishes
Speaker 1: free riders, which means that resentment is deeply ingrained. Free
Speaker 1: riders violate a basic sense of fairness that we hold dear.
Speaker 1: The trouble is when people sense free riders in their
Speaker 1: midst they tend to cut off their contributions, so everyone loses.
Speaker 1: We have a lot to overcome if we're going to
Speaker 1: take on our existential risks. On the next episode of
Speaker 1: the End of the World with Josh Clark, it is
Speaker 1: possible to take something which is not really part of
Speaker 1: common sense morality, and then within a generation, children are
Speaker 1: being raised everywhere with this as part of just a
Speaker 1: background of beliefs about ethics that that they live with.
Speaker 1: So I really think that we could achieve that there
Speaker 1: is hope