Speaker 1: Brought to you by Toyota. Let's go places. Welcome to
Speaker 1: Forward Thinking. Hey everyone, and welcome to Forward Thinking, the
Speaker 1: podcast that looks at the future and says I swung
Speaker 1: the election. Friends, that's no small order. I'm Jonathan Strickland
Speaker 1: and I'm Joe McCormick, and I have a question for you, guys.
Speaker 1: Have you ever googled yourself to be met with horror?
Speaker 1: Not because of something that you did that's chronicled on
Speaker 1: the Internet in an embarrassing way, but because somebody has
Speaker 1: the same name as you, and they're higher in Google
Speaker 1: results than you are, and they do something that you
Speaker 1: don't necessarily want people thinking that you do. It is
Speaker 1: very difficult for me to answer this question, uh, without
Speaker 1: putting my own person philosophy on the line. I'll just
Speaker 1: say yes, uh, speaking as from what I can tell,
Speaker 1: the one end only Lauren Vogel baumb on this planet.
Speaker 1: That's a lucky condition. Yeah, I actually don't know how
Speaker 1: that feels. Well, there's somebody with the same name as
Speaker 1: me who's some kind of erotic photographer. Well, let me
Speaker 1: just say this. There is a certain conservative senator from
Speaker 1: Texas who probably is irritated that he has the same
Speaker 1: name that I do. Really like he's tired of getting
Speaker 1: emails about how this new app wor let me say, actually,
Speaker 1: I say Senator. I think he's the State House of Representatives,
Speaker 1: so I believe I misspoke. But yes, there is a
Speaker 1: politician in Texas who has the same name as I do,
Speaker 1: and I'm sure he is endlessly irritated by my tweets, etcetera.
Speaker 1: Why are people asking me how Mulnier works? How do
Speaker 1: you say that mull mullner? Okay, well, this is going
Speaker 1: to be related to the topic that we're going to
Speaker 1: talk about today, which is Google rankings, or i'd say
Speaker 1: more broadly search rankings. Though let's be honest, what we're
Speaker 1: really talking about is Google the United States. Sure, sure,
Speaker 1: we read this really good article in Wired by one
Speaker 1: Adam Rogers. It's called Google Search Algorithm could Steal the Presidency,
Speaker 1: and we found it so interesting because in it, Rogers
Speaker 1: introduces a concept that he calls Google mandering. Yeah what
Speaker 1: google mandering? As in Google mandering and an spirit of
Speaker 1: Google has appeared with us here in the requests. I
Speaker 1: thought it was an introductory class at Hogwarts. You know,
Speaker 1: you had your herbology and google mandering. No, no, but
Speaker 1: based on jerrymandering, which of course is the practice of
Speaker 1: rigging a election populations to get the results that you're
Speaker 1: looking for. Right, So, maybe if you have a lot
Speaker 1: of people who you're expecting to vote against your party,
Speaker 1: you can just safely confine them all to one weirdly
Speaker 1: shaped district so they're not going to be threatening to
Speaker 1: vote in your district. Yeah. The the basic idea here
Speaker 1: is that the way that search results could be displayed
Speaker 1: might influence someone's decision on an important film thing like
Speaker 1: who to vote for in an election. What that's so crazy,
Speaker 1: you guys. Google surely cannot influence something so personal is
Speaker 1: voting decisions, or something so complex as election results. Right. Well,
Speaker 1: so here we're gonna get into something that is sort
Speaker 1: of a requires some standing back and analyzing the way
Speaker 1: media affects our perception. Now, the way media affects our
Speaker 1: voting is pretty obvious in one sense, as in, you
Speaker 1: can read articles, or watch television or get any other
Speaker 1: kinds of sources of entertainment or information that tend to
Speaker 1: favor one side of a debate debate over the other side.
Speaker 1: But there's a different way that your media can affect
Speaker 1: to your decisions, and it can be at the level
Speaker 1: above that, not just that you're looking at one article
Speaker 1: and it and it consistently favors one side over another,
Speaker 1: but in the selection or availability of sources available to you. Yeah.
Speaker 1: So in other words, uh, I mean, well, let's put
Speaker 1: to it a a very simple way. If you lived in
Speaker 1: a very remote location and only one newspaper ever arrived,
Speaker 1: and that newspaper had a very specific slant political slant,
Speaker 1: and that's all the information you get, it would be
Speaker 1: very difficult for you to make an unbiased decision if
Speaker 1: you can only get the anarchist world news today, in
Speaker 1: which case you probably just don't vote, but but you
Speaker 1: don't vote with authority. But yeah, that's that's a that's
Speaker 1: an example. So now, when we talk about the the
Speaker 1: use of online sources and also just just the various
Speaker 1: media sources that are out there, we're not necessarily boiling
Speaker 1: it down to something that's simplistic, but the point being
Speaker 1: that when you get one of these influential voices to
Speaker 1: come into an area, there appears to be a measurable
Speaker 1: impact to that. Yeah, and that's not you don't have
Speaker 1: to take our word for it. Um, there's uh, there
Speaker 1: is a actual phenomenon called the Fox News effect. Well no, okay,
Speaker 1: So to be clear, we're not going to be taking
Speaker 1: a political stance one way or another. This is about
Speaker 1: the measurable effects of media availability in surgeons. Yes, yes,
Speaker 1: uh so. The Fox News channel, as you may may
Speaker 1: not know, started up in October, wherein it joined other
Speaker 1: twenty four hour cable news channels like CNN and MSNBC.
Speaker 1: And you know, okay that the channel has long maintained
Speaker 1: to have no bias towards anything other than fairness. Um,
Speaker 1: but it's always been on the conservative side of fairness. Yeah,
Speaker 1: I think everybody is aware of this. Different channels tend
Speaker 1: to have different political leanings. You can, I can expect
Speaker 1: MSNBC to be more to the left, you can expect
Speaker 1: Fox News to be more to the right. Sure. So
Speaker 1: it was introduced into the cable packages of some likewent
Speaker 1: of towns in the United States between November of two thousand,
Speaker 1: which was the year of the Bush versus Gore versus
Speaker 1: Ralph Nader kind of sort of presidential election, and a
Speaker 1: group from the National Bureau of Economic Research, which is
Speaker 1: this nonprofit and hypothetically nonpartisan organization that studies the economy
Speaker 1: and politics saw in this an opportunity for learning specifically
Speaker 1: about media bias and putting some numbers into it. So
Speaker 1: they gathered voting data from over nine thousand towns and
Speaker 1: they found that the Republican Party had gained a zero
Speaker 1: point for to zero point seven percentage points in towns
Speaker 1: that had gained access to Fox News and and furthermore,
Speaker 1: that the channel had encouraged voter turnout there. Their estimates
Speaker 1: are that channel convinced some like three to eight percent
Speaker 1: of its viewers to vote Republican, which which sounds like
Speaker 1: a small amount, but that is more than enough to
Speaker 1: have included like ten tho voters in Florida, which is
Speaker 1: enough to have flipped the state, which was the decider
Speaker 1: in the year two thousand election certainly was yeah, so
Speaker 1: everything turned on Florida. That's where we had the recount,
Speaker 1: and we eventually had to have the U. S. Supreme
Speaker 1: Court step in and say, okay, we're putting an end
Speaker 1: to this. Congratulations Captain Bush. And to be clear, the
Speaker 1: three to eight percent that were convinced to to vote Republican,
Speaker 1: many of those could have been self identifying Republicans. Well,
Speaker 1: before the Fox News channel came on, they were just
Speaker 1: you know, convinced to go out and actually cast a
Speaker 1: vote partially because of Fox News. So it's not to
Speaker 1: say that, you know, and when a media uh company
Speaker 1: of some sort enters into a region that it magically
Speaker 1: changes uh three to eight percent of the population to
Speaker 1: that side. You know. I think usually those estimates are
Speaker 1: referring to um to undecided voters. Yeah, And the thing
Speaker 1: that's interesting here, I think is that this is a
Speaker 1: comment about the effects of the availability of different types
Speaker 1: of sources. So it's not necessarily that in these towns
Speaker 1: where suddenly they got access to Fox News on cable,
Speaker 1: everybody was sat was you know, made to sit in
Speaker 1: a room and forced to watch Fox News. But now
Speaker 1: you have these sources of available, some people are going
Speaker 1: to consume those sources. And it appeared to have some
Speaker 1: effect on how people voted. But there are other ways
Speaker 1: that selection of messaging in media can have an effect
Speaker 1: on voter turnout and on elections. Yeah, so you're referring
Speaker 1: specifically to a campaign that Facebook ran to to inspire
Speaker 1: people to go out and vote. This is great because
Speaker 1: in this case, it is a supposedly entirely neutral message.
Speaker 1: It doesn't say go vote for my anarchist candidate clause
Speaker 1: mcgrabby clause Yeah, that's why I would vote in that
Speaker 1: anarchist party. Yes, absolutely no. No, Facebook, in this experiment,
Speaker 1: which Facebook sometimes runs experiments on you guys, I just
Speaker 1: you should know that, just flat out to start with,
Speaker 1: it's pretty cool. You're helping science, whether you want to
Speaker 1: or not. Yeah, you're a product and a science experiment. No, No,
Speaker 1: it's it's great though. I mean because because they were
Speaker 1: able to gain a sample of sixty one million users
Speaker 1: who were eighteen or older who accessed the site on
Speaker 1: the day of the congressional election in and and there,
Speaker 1: they just put out these messages about going out and voting.
Speaker 1: It wasn't again, it wasn't voting for a particular person. Uh.
Speaker 1: It was just either either go vote or go vote
Speaker 1: and check out how many other friends of yours have
Speaker 1: gone and voted. So the the experiment split their overall
Speaker 1: sample into three groups. One percent about uh six hundred
Speaker 1: and eleven thousand users was a control group that actually
Speaker 1: received no message. Another one percent received a purely informational
Speaker 1: message at the top of their news feed. That just
Speaker 1: encouraged them to vote, linked to info about local polling spots,
Speaker 1: provided an optional voted button to click, and and gave
Speaker 1: account of other Facebook users who had clicked on it.
Speaker 1: Cent of the sample, about sixty million users got all
Speaker 1: of that informational stuff, plus a little social message that
Speaker 1: included the profile picks of up to six of their
Speaker 1: Facebook friends who had clicked the I Voted button. And
Speaker 1: they were furthermore that the researchers were furthermore able to
Speaker 1: match six point three million of those users with public
Speaker 1: voting records to see whether their messages had affected voting practices.
Speaker 1: And I mean they did well. Okay, they found that
Speaker 1: the informational message actually had no effect, but but the
Speaker 1: social message made people point three percent more likely to
Speaker 1: click through to polling information and point four percent more
Speaker 1: likely to actually go and vote. And that might sound
Speaker 1: like a tiny effect, but remember that we're dealing with
Speaker 1: those those millions and millions of study participants. So the
Speaker 1: researchers estimate that about three hundred and forty people went
Speaker 1: to the polls who otherwise would not have gone because
Speaker 1: they saw that social message. Yeah. The interesting thing to
Speaker 1: me here, besides the fact that there was a noticeable
Speaker 1: effect of this approach is that Facebook could very easily
Speaker 1: sway an election simply by sending the go vote message
Speaker 1: to people that had been identified as being sympathetic towards
Speaker 1: the mission statement of Facebook. So in other words, that
Speaker 1: Facebook has particular uh policies that they really want past,
Speaker 1: and there are particular politicians as associated with those policies.
Speaker 1: And because we share everything we have on Facebook, we
Speaker 1: we tell Facebook everything our deepest, darkest secrets, we whisper
Speaker 1: into its ear, they can identify with pretty high precision
Speaker 1: which people would be the most sympathetic toward the candidates
Speaker 1: that they themselves would want to support. Or, if you
Speaker 1: want to get conspiratorial, in another direction, you could say
Speaker 1: that Facebook could potentially sell this service to a candidate.
Speaker 1: So maybe if suddenly clause Mate gravity Claus is flush
Speaker 1: with cash, he could go and pay Facebook to just
Speaker 1: tell people who have said supportive things about the Anarchist
Speaker 1: Crab Party to go vote on election day, but not
Speaker 1: to tell anybody else to go vote, right, So, in
Speaker 1: other words, the ones who are most likely to support
Speaker 1: that message get the extra incentive or the extra encouragement
Speaker 1: to go and vote. The other people don't get that
Speaker 1: in uragement, and ultimately Facebook could say, look, we didn't
Speaker 1: tell them who to vote for. We didn't tell the
Speaker 1: other people not to vote. All we did was Santana
Speaker 1: vote message. That's all we said. And it ends up
Speaker 1: being this. You know, it's it's almost like all we
Speaker 1: had to do was put the idea into people's heads
Speaker 1: and stand back and let the rest happen. Now, that's
Speaker 1: about a social networking platform. That's something that could potentially happen,
Speaker 1: although you could probably figure that out, like if you
Speaker 1: were talking about numbers large enough where that sort of
Speaker 1: thing would probably become a parent pretty quickly and I'm
Speaker 1: guessing would not reflect too well upon said social network.
Speaker 1: Let's talk about search engines and and why they would
Speaker 1: matter in an election is just that people use them
Speaker 1: to do research these days. Uh. For example, in the
Speaker 1: days leading up to end just following the presidential election,
Speaker 1: Google users interest in search terms related to Romney reached
Speaker 1: the highest that they have ever been by far are
Speaker 1: and terms related to Barack Obama spiked higher than they
Speaker 1: had since his initial election in two thousand eight. So
Speaker 1: people were they were cramming for the final So that's
Speaker 1: the thing. Right. If if people are turning to search
Speaker 1: engines in order to get information, and we've already established
Speaker 1: that the types of information you get can influence your decisions,
Speaker 1: it stands to reason that the search engines results page
Speaker 1: has to be pretty important. Yeah, but okay, let me
Speaker 1: play the dumb guy here. Okay, sure, no, not the
Speaker 1: dumb guy. Let me play the guy with a reasonable concern. Okay,
Speaker 1: reasonable concerned guy. The Internet is a democratizing force, because
Speaker 1: isn't it great that you can go into a search
Speaker 1: engine and you can get information that represents all the
Speaker 1: points of view out there. You can find articles that
Speaker 1: are pro Barack Obama, you can find articles anti Barack Obama,
Speaker 1: you can find pro Mitt Romney, you can find anti
Speaker 1: Mitt Romney. And you might only be able to find
Speaker 1: articles that are pro Clausemate Grabby Clause. That's because he
Speaker 1: is so wonderful. But the point is, whatever information people
Speaker 1: want to present, whatever opinions they want to publish, you
Speaker 1: can find it on the Internet. So why why wouldn't
Speaker 1: the Internet be a perfectly neutral source to get your
Speaker 1: information from? Well, first, that works under the assumption that
Speaker 1: every single web page out there is treated equally by
Speaker 1: search engines. Ah, and we know that not only is
Speaker 1: that not the case, it in fact cannot be the
Speaker 1: case because how would you present every single potential return
Speaker 1: on any given query where they're all ranked at the
Speaker 1: same level. Would you just have an infinitely scrolling bar,
Speaker 1: would you just have a camera view that is constantly
Speaker 1: hovering over different UH titles? And even so, then what
Speaker 1: order do you put them in? Because that alone creates
Speaker 1: some sort of sense of rank. Now you could say, well,
Speaker 1: wait a minute, what if we randomize search results so
Speaker 1: that we and nothing gets prefering chilly treated and featured
Speaker 1: towards the top. But that would sort of go exactly
Speaker 1: at cross purposes of what somebody like Google is trying
Speaker 1: to do, where they're constantly trying to give you the
Speaker 1: best possible result for whatever terms you entered. Yeah, that
Speaker 1: that's why we go to Google more often than to
Speaker 1: other search engines, because it more frequently returns us links
Speaker 1: that were interested in, right, so that it would go
Speaker 1: entirely against their interests and to be frank, our interests
Speaker 1: if they were to say, well, let's randomize all of
Speaker 1: the different articles about Barack Obama and Mitt Romney so
Speaker 1: that none of them get preferential treatment. Randomization would just
Speaker 1: mean that for Because let's go outside of the elections
Speaker 1: for a second. You don't want random results for any
Speaker 1: given query. When you put in a query. What you're
Speaker 1: the what you want to see on that first page
Speaker 1: of results, which I'll get to in a second, is
Speaker 1: a link to a site that answers the question you have,
Speaker 1: gives you the information you want, links to the restaurant
Speaker 1: you're looking up, whatever it may be. That's what you get.
Speaker 1: You aren't You don't want a random thing that happens
Speaker 1: to relate tangentially or otherwise to whatever it was you
Speaker 1: were looking for. So, because of Google's uh uh efficient
Speaker 1: means of returning search results, it has shaped our behaviors online.
Speaker 1: And this is something that we've seen over and over
Speaker 1: again in various studies. So most of us don't bother
Speaker 1: to look beyond the first page of search results for
Speaker 1: any given query. Yeah, most of us, And most of
Speaker 1: us put in a search query. Well, and you don't
Speaker 1: want to write you have better things, Yeah, you have
Speaker 1: better things to do than to go through eighteen pages
Speaker 1: of search results trying to find one that is the
Speaker 1: closest and most relevant to to what you are searching for,
Speaker 1: at least if it's something you know, especially something casual,
Speaker 1: right like I don't. I don't want to spend twenty
Speaker 1: five minutes going through to find uh, something like I'm
Speaker 1: looking up shoes. Put that on number one. You know,
Speaker 1: in my experience as someone who has been doing various
Speaker 1: kinds of deep research on the web for many years now,
Speaker 1: I can say that I think the organization and prioritization
Speaker 1: of Google search results has gotten demonstrably better over the
Speaker 1: past ten years or so. It used to be much
Speaker 1: more common that to find the ideal example of the
Speaker 1: thing I'm looking for, I would have to go deep
Speaker 1: into other pages. That happens way less now. Now It's
Speaker 1: it's way more common that exactly the thing I'm looking for,
Speaker 1: or the best example of the thing I'm looking for
Speaker 1: available on the web is on the first page, which
Speaker 1: is both good and bad. It's good in the sense
Speaker 1: that we're finding the stuff we want more quickly than before.
Speaker 1: It's bad and that if you are doing research into
Speaker 1: a topic and you're unfamiliar with that topic, you have
Speaker 1: been conditioned to go after those first few links, and
Speaker 1: it may behoove you to go deeper to find to
Speaker 1: get a full understanding of whatever topic it is that
Speaker 1: you're you're researching. Um. But see, I mostly do research
Speaker 1: on academic subjects. And this is also problematic because if
Speaker 1: you're looking at academic subjects, most of the time, not always,
Speaker 1: but most of the time, you're looking at a kind
Speaker 1: of objective approach to whatever the subject matter is. And
Speaker 1: most of the subjects that you're researching are fairly objective
Speaker 1: to begin with. I mean, they're they're not something so
Speaker 1: hot button as as political stuff, right, So if I
Speaker 1: were doing searches on political stuff, it may be that
Speaker 1: I would discover more of leaning on one side or
Speaker 1: the other. I mean that's a possibility. I just haven't
Speaker 1: done that. That's not what I use Google for. Well, yeah,
Speaker 1: and if you do go deep into the search results
Speaker 1: on certain political candidates, you'll find that there's tons of
Speaker 1: garbage out there that's not going to be useful to
Speaker 1: anybody on either side unless you're just looking for some
Speaker 1: slander to rile you up. Sure, sure, well, And and
Speaker 1: that's for that's for relatively you know, popular terms that
Speaker 1: a lot of people are going to be writing about.
Speaker 1: That The the problem that I wind wind up running
Speaker 1: into with Google is that I don't know, you know,
Speaker 1: I do so much research. I start so much research
Speaker 1: there that I'd say that about like once every week
Speaker 1: or two, I wind up having to go through some
Speaker 1: like dozen iterations of a search term or or dig
Speaker 1: like several pages back in the search results just to
Speaker 1: find what I'm looking for. Partially probably because of the
Speaker 1: human error of me looking for information on a topic
Speaker 1: that I know nothing about and thus don't know the
Speaker 1: right terms to search with. But sometimes it's kind of like, no, Google,
Speaker 1: I did not want to know about specialty cheese shops
Speaker 1: in my area. When I searched for active cheese culture,
Speaker 1: you find like the neighborhood where all the joggers go
Speaker 1: to the cheese shop. Yeah, that's the active cheese culture
Speaker 1: in my neighborhood. Yeah, Or is it the one where
Speaker 1: the cheeses are all living in a culture where they're
Speaker 1: active like Fraggle Rock. I encounter the same thing all
Speaker 1: the time researching for podcasts. In fact, in fact, the
Speaker 1: the other topic that we will be recording today. As
Speaker 1: we're sitting in the studio, I was having issues with
Speaker 1: that where I kept on going back and putting in
Speaker 1: different search terms because I thought there's got to be
Speaker 1: a study about this, and I could not find something. So,
Speaker 1: I mean, this is a common issue. So that's also
Speaker 1: something that take into account, is that the search results
Speaker 1: will be as close to relevant as possible. It behooves
Speaker 1: Google to have that. If if Google was seen as
Speaker 1: being an unreliable source for relevant information, people wouldn't use Google.
Speaker 1: And that's where Google's value is. So they have they
Speaker 1: have a an incentive to do that. However, Yeah, so
Speaker 1: let's look at the ways then the search results actually
Speaker 1: do matter in practice. Yeah. Sure, because again, you don't
Speaker 1: have to take our word for it. People, people have
Speaker 1: done actual research on how likely people are to click
Speaker 1: past the first page of Google results. Done at all
Speaker 1: tell me only what percentage of clicks go to the
Speaker 1: first page. So more than nine out of ten clicks
Speaker 1: in Google results are going to happen out of the
Speaker 1: first page. Yeah, Or if you want to think of
Speaker 1: it another way, fewer than one out of ten will
Speaker 1: go to the second page or beyond. Yeah, So it's
Speaker 1: not just to the second page, it's to all other
Speaker 1: results in the entire web come by. Yeah, when you
Speaker 1: see like four point eight trillion results returns from for
Speaker 1: your search query, nine five percent of the clicks are
Speaker 1: going on that first page and not even on that
Speaker 1: first page, right, Yeah, thirty two point five percent of
Speaker 1: them are going straight to that first result. Now is
Speaker 1: this including the sponsored ad results? This is going to
Speaker 1: the I believe it's the non sponsored ones, So sponsored
Speaker 1: ones also take up some of that percentage. Obviously, the
Speaker 1: second result gets seventeen point six percent, So clearly first
Speaker 1: place is the place to be. That's where you're going
Speaker 1: to get the most traffic um And also, if you
Speaker 1: look at the last results on the first page of
Speaker 1: a search engine results page, why would you ever scroll
Speaker 1: all the way down to the last result on the
Speaker 1: first page. Well, very few people do, as a matter
Speaker 1: of fact, but those who do still greatly outnumber those
Speaker 1: who go to the second those who go to the
Speaker 1: second page. So yeah, if you look at the number
Speaker 1: of people who click on the the final result on
Speaker 1: the first page of a search engine results page, it's
Speaker 1: more people than those who click on the first result
Speaker 1: of the second page. So if you're somebody who's putting
Speaker 1: content on the web, whether whether you're trying to monetize
Speaker 1: it as a business like you're writing articles and make
Speaker 1: your money through advertising, you need clicks to keep going,
Speaker 1: or whether you're just trying to get your message out
Speaker 1: in one way or another. If you want people to
Speaker 1: see your page, being on the first page of search
Speaker 1: results is crucial. It's a huge help. I mean, it's
Speaker 1: one of those things that we always here. Is becoming
Speaker 1: less important because of the rise of the importance of
Speaker 1: social networking, but it's still easily one of the best
Speaker 1: ways to drive huge amounts of traffic. If you what
Speaker 1: people call own a search term. If you own a
Speaker 1: search term, then you become the destination for everyone who
Speaker 1: searches for that because if you're if you're number one
Speaker 1: on that list, you're getting of that traffic. If it's
Speaker 1: a popular search term that translates into hundreds of thousands
Speaker 1: of page views, that's a big deal. But what if
Speaker 1: the search term you own is somebody else's name, and
Speaker 1: that person's name that's the name of a political candidate,
Speaker 1: and you don't like that political candidate. Well, if if
Speaker 1: you are the go to source for that, then that
Speaker 1: number one result could be a very negative uh portrayal
Speaker 1: of said candidate. And that's just that's just the way
Speaker 1: it is, although if you're in Europe there's actually some
Speaker 1: way of getting around that, but that's you're in the
Speaker 1: United States. That down here in the United States, this
Speaker 1: worked out to particularly unfortunate effects for one Republican contender
Speaker 1: in recent years. Y'all, I'm sure aware of the Rick
Speaker 1: Santorum effect. We don't need to go into detail about
Speaker 1: the about the Santorum web patrolling that occurred around this dude. Yeah, yeah,
Speaker 1: if you are interested in reading up on that and
Speaker 1: don't mind some kind of gross subjects, you can go
Speaker 1: right ahead, right So you know, we we've seen that
Speaker 1: there are there are huge incentives for anyone who's going
Speaker 1: to be using search engine traffic to drive whatever it
Speaker 1: is they're doing, whether it's a political campaign, a company
Speaker 1: and organization, whatever it may be, there's a huge incentive
Speaker 1: for them to try and get on that first page
Speaker 1: because being anywhere else, you might as well not even
Speaker 1: worry about search engine traffic. You've got to concentrate on
Speaker 1: something else you want to you have to be on
Speaker 1: that first page to to leverage traffic. Sure, but getting
Speaker 1: onto that first page is a really tricky game. Yeah,
Speaker 1: because it's not like it was in the old days.
Speaker 1: Like in the old days with search engines, one of
Speaker 1: the things you could do is you could pepper your
Speaker 1: page with tons of irrelevant metadata that had nothing to
Speaker 1: do with your content. You know, I this page is
Speaker 1: an ad for the virtues of grabby or what's his name?
Speaker 1: Clause clause. But also you might like this page if
Speaker 1: you're interested in metallica, bckney spears, right, yeah, what were
Speaker 1: the other things popular in the early two thousands, Yeah, yeah,
Speaker 1: So if you've ever gone to well, those old web
Speaker 1: pages where there's just a a just a garbage pile
Speaker 1: of unrelated words at the bottom of the page, or
Speaker 1: sometimes it's hidden where they made the text the same
Speaker 1: color as the background. That was an attempt to game
Speaker 1: the system to get on that first page by not
Speaker 1: only trying to get you know, look like you're an
Speaker 1: important page, but also just throw in so many search
Speaker 1: terms that no matter what someone was searching, they would
Speaker 1: end up at your page. Never mind the fact that
Speaker 1: the information on that page may be completely irrelevant to
Speaker 1: the search query, that no one cared about that they
Speaker 1: wanted the page use. They want that little page counter
Speaker 1: that was on everybody's homepage way back in the day
Speaker 1: to click up a notch, Um, but come for the
Speaker 1: false advertising, stay for the dancing babies. Yeah. Now, eventually
Speaker 1: search engine algorithms got more sophisticated than that. Engineers built
Speaker 1: better algorithms that would ignore this metadata. In fact, today's
Speaker 1: Google algorithms that Google says that metadata plays like no
Speaker 1: role at all in page ranking, that it's more important
Speaker 1: that the page actually contain relevant information that is key
Speaker 1: to whatever the query was, and part of that is
Speaker 1: decided based on how many other pages linked to this page. Yeah,
Speaker 1: so if I'm doing a search for people can try
Speaker 1: to game that too. But yes, but first, let's say
Speaker 1: that I'm doing a search for a particular candidate. Um,
Speaker 1: and I'm not going with yours because I'm I'm frankly,
Speaker 1: I'm I'm the I'm in the anti crab party. I'm
Speaker 1: more of a lobster party. I will go with Pincheo
Speaker 1: Hula Hans. So you're against the crab anarchist party, you're
Speaker 1: in the lobster fascist party. Fascist is such a such
Speaker 1: a such a cruel word. I think of them as
Speaker 1: the imperialists. So law an order party. Yeah, So so
Speaker 1: Pinchio Hulahan, let's say, let's say, um, you know, I
Speaker 1: want to do a search on pinchio hula haan. Well,
Speaker 1: the website that comes up first maybe one that lots
Speaker 1: of other websites linked to whenever they are commenting upon
Speaker 1: pinchio hulahan. So there might be news stories that link
Speaker 1: back to this website, and there might be lots of
Speaker 1: other blog pages, all sorts of stuff. The way page
Speaker 1: ranking works in general is that incoming links are worth
Speaker 1: a certain amount. Also, they're weighted, so incoming links from
Speaker 1: more important pages are worth more. So in other words,
Speaker 1: like a big um news outlet like seeing then if
Speaker 1: they link out, that link out is worth more than
Speaker 1: Joe Bob's blobster blog. Bob blah blahs law blog is
Speaker 1: not going to be as big as CNN, so uh,
Speaker 1: you know those are both factors in it now. People
Speaker 1: tried to game the system too. There were a lot
Speaker 1: of I mean, you probably have encountered this where you've
Speaker 1: done Oh yeah, you go to you you find a
Speaker 1: you know, you do a search for something, something pops up,
Speaker 1: and you know you click on the first result because
Speaker 1: ninety one point or thirty two point five percent of
Speaker 1: us do that, So you click on that first result,
Speaker 1: and what it takes you to is just a list
Speaker 1: of links, and all it is is just links for days,
Speaker 1: all the way down the page. People tried to make
Speaker 1: these link farms in order to game the system and
Speaker 1: build up other page ranks of other sites. But you know,
Speaker 1: eventually algorithms got fisky enough to see that too. Yeah.
Speaker 1: In one entertaining and slightly less soul sucking example that
Speaker 1: there was one time, very briefly back in the days
Speaker 1: that I was on live journal, a uh an instance
Speaker 1: of Neil Gaiman attempting to get the search term Pendelotte
Speaker 1: to lead to Neil Gaiman's website, and he succeeded because
Speaker 1: he's Neil Gaman, he owns the internet. Why Penjialette because
Speaker 1: he had some like some like temporary jokey feud with Pendulot.
Speaker 1: He was like, hey, but I can do this weird
Speaker 1: thing to you, and Pendulotte was like, no, you can't.
Speaker 1: And that's my pendulout impersonation. I'm not sure it's very quality, yeah, yeah, yeah, yeah,
Speaker 1: it's got a kind of a grally voice. Yeah. And
Speaker 1: then I mean that was one of those things where
Speaker 1: there were it was easier to game the system, it's
Speaker 1: harder to do it now, uh these days. The way
Speaker 1: Google pitches it anyway, is that in order to get
Speaker 1: on that first page of results, you need to demonstrate
Speaker 1: that the site that that page belongs to is dependable,
Speaker 1: that the information inside is relevant to the search query,
Speaker 1: that it's of a high quality. These are all, you know,
Speaker 1: qualities that are difficult to It's difficult to say how
Speaker 1: they measure that. Google doesn't make their algorithm public by
Speaker 1: the way they it's it's secret sauce. I mean, stuff
Speaker 1: like that sounds highly subjective to me, So it's hard
Speaker 1: to know how they could really verify that. Well sure, yeah,
Speaker 1: but but okay, So so either way, there are all
Speaker 1: these algorithms at work that are choosing what we see
Speaker 1: when we open up a Google search, and you know,
Speaker 1: depending on what the content of those articles is, we're
Speaker 1: we're going to run into some bias on that first
Speaker 1: term that we click on. Yeah, and in fact, if
Speaker 1: we were to be a little unethical, let's say that
Speaker 1: we're in charge of an enormous search engine and we
Speaker 1: can actually tweak things so that very specific search we
Speaker 1: we can rank essentially dynamically whatever the pages are and
Speaker 1: present the links that are most in line with our
Speaker 1: own worldview as the top ranking ones, and put the
Speaker 1: ones that maybe argue with our worldview further down. We
Speaker 1: already know five of everyone who goes to that search
Speaker 1: engine results page is clicking on that first page of results.
Speaker 1: So yeah, if the president of Arthropod Google prefers, you
Speaker 1: might very well get a first page that's heavy on
Speaker 1: the pro pinchy literature. But I mean, so we're not
Speaker 1: saying that we think Google is doing this, but the
Speaker 1: potential is there and it's, as it turns out, pretty
Speaker 1: well research. Yeah, so this is going to be the
Speaker 1: core of this episode. Here is this paper by a
Speaker 1: couple of guys named Robert Epstein and Ronald E. Robertson.
Speaker 1: They address this effect that they call the search engine
Speaker 1: manipulation effect and it's possible impact on the outcomes of elections.
Speaker 1: That was the title of the paper in paper that
Speaker 1: they released through the Proceedings of the National Academy of
Speaker 1: Sciences or p n a S. And in this paper
Speaker 1: they review the results of five different experiments carried out
Speaker 1: in two countries testing the effects of search engine results
Speaker 1: on subjects voting preferences. Right, So, They were working with
Speaker 1: the hypothesis that the search engine results could affect a
Speaker 1: person's decision making uh, just based upon which results are
Speaker 1: presented first, and that it could affect enough people that
Speaker 1: it could in fact sway an election. Right. So they
Speaker 1: started in San Diego. They started small, to just test
Speaker 1: the hypothesis on a relatively tiny sample size. They gathered
Speaker 1: a hundred and two volunteers. They were the volunteers were
Speaker 1: actually paid for their participation in the experiment. All the
Speaker 1: volunteers in the experience that we're talking about gotam tiny payment.
Speaker 1: I think in the first one it was twenty five dollars,
Speaker 1: but in subsequent ones it was lower. Uh. The volunteers
Speaker 1: were presented with two candidates for a election which had
Speaker 1: already happened in Australia. In Australia, these are people in
Speaker 1: San Diego given two candidates for Prime Minister of Australia
Speaker 1: in twenty um. And the two candidates were Tony Abbott
Speaker 1: and Julia Gillard. And the volunteers were divided into three
Speaker 1: test groups. So what the researchers wanted to do was
Speaker 1: present the group with choices that they weren't likely to
Speaker 1: have previous knowledge about They wanted to They wanted to say,
Speaker 1: all right, well, here are these two candidates that we
Speaker 1: are fairly confident most of the people that we've gathered
Speaker 1: have either never heard of or they've heard very little
Speaker 1: about them. Americans don't tend to have a lot of
Speaker 1: opinions about Australian politics, if they even have opinions about
Speaker 1: their own. Sorry, that was that was bleak. But but
Speaker 1: they but they did want to choose actual politicians because
Speaker 1: they wanted uh, people for whom they are already robust
Speaker 1: Google search results. Yes, they wanted to use real articles
Speaker 1: that were written about these two people rather than have
Speaker 1: to write up a bunch of fake ones. So they
Speaker 1: used real articles, a big collection of them, and um
Speaker 1: and some of the articles put one candidate in a
Speaker 1: favorable it's over the other, and other articles were the opposite.
Speaker 1: They some favorite Abbots, some favorite Gillard, and then they
Speaker 1: also had some articles that they classed as neutral. They said,
Speaker 1: you know, weren't really biased one way or another. So
Speaker 1: they then went forward with a double blind test. I'll
Speaker 1: talk about that in a second. But the researchers first
Speaker 1: asked each group which of the two candidates they would
Speaker 1: most likely support an election based on just a very
Speaker 1: brief biography of each candidate that was presented in as
Speaker 1: non biased away as possible. And they said that there
Speaker 1: wasn't really any um measurable difference between the two candidates.
Speaker 1: People didn't really care going in. Yeah, it was like
Speaker 1: essentially a coin flip situation. And then each group had
Speaker 1: the chance to do research on the two candidates using
Speaker 1: a mock search engine called cadoodle in the experiment. Yeah,
Speaker 1: much better algorithm than Arthur pod Google. Uh. Anyway, I'm
Speaker 1: surprised they didn't have Pring or something, but no, it's
Speaker 1: it's cadoodle. So you would use cadoodle to do a
Speaker 1: you know, some search on these You would put in
Speaker 1: the names of the candidates and the search results would
Speaker 1: pop up and um uh. They divided the group in
Speaker 1: or groups into three. They used a double blind approach,
Speaker 1: which meant that neither the researchers nor the participants had
Speaker 1: any knowledge of what the hypothesis of the study was,
Speaker 1: so they didn't know why they were doing this. This
Speaker 1: is a common thing in in research like this to
Speaker 1: try to ensure that your researchers or the people working
Speaker 1: on the experiment aren't giving subtle cues to the participants
Speaker 1: what they should do. Accidentally by said, right, because most
Speaker 1: of the time participants want to please we as humans
Speaker 1: want to please everyone, right yeah, so yeah, And because
Speaker 1: this is all about studying bias in the first place,
Speaker 1: you don't want to corrupt that with introducing other bias.
Speaker 1: Than you can't really see what the results of your
Speaker 1: intended bias was. So uh so, none of neither the
Speaker 1: participants or the people administering this had any knowledge of
Speaker 1: what the hypothesis was, or what group anyone was assigned to,
Speaker 1: or what the purpose of the other groups happened to be.
Speaker 1: So so what have and when these groups did their
Speaker 1: cadoodle searches, Well, you had one group that received absolutely
Speaker 1: neutral results. There were just it was just like a
Speaker 1: regular search results page, where no candidate was given favorable treatment.
Speaker 1: Then another group would get favorable results for one of
Speaker 1: the two candidates. The other group got favorable results for
Speaker 1: the other of the two candidates. And the way this
Speaker 1: worked is that they had multiple pages that they could
Speaker 1: click on if they wanted, so that you'd get like
Speaker 1: six pages of results, and you could review as much
Speaker 1: of the results as you wanted to and and Also,
Speaker 1: all of the three groups had the same collection of
Speaker 1: articles at their disposed, right, it was just in a
Speaker 1: different order, exactly. Yeah, So one group might get this
Speaker 1: staggered kind of response where it's favorable to one candidate,
Speaker 1: then favorable to another, and then back and forth. Another
Speaker 1: one might get all of the you know, pro Abbot
Speaker 1: articles at the beginning, and then only get to the
Speaker 1: pro Guillard articles at the very end Gillard or Gillard
Speaker 1: how we pronounce that, And then the other group would
Speaker 1: get it the other way around. So, uh, you know,
Speaker 1: they again, no one knew what was going on or
Speaker 1: was told what was going on at the beginning. Uh,
Speaker 1: they did. The researchers did decide that they had to
Speaker 1: insert a question to find out if people were picking
Speaker 1: up on what was going on. But they had to
Speaker 1: be very careful with how they asked that question, right,
Speaker 1: because if you ask the people, did you find these
Speaker 1: search results biased? There, you're kind of giving them a
Speaker 1: clue they should be interpreting what's going on. So how
Speaker 1: do you ask, hey, do you think we totally rigged
Speaker 1: the search results? They they The way they did was
Speaker 1: they said, did anything bother you? It was vague enough
Speaker 1: where they felt you know, maybe if no one is
Speaker 1: really paying attention, they'll just say no nothing bother me.
Speaker 1: It would only be the people who are really focusing
Speaker 1: that said yeah, I happened to notice something weird. Uh.
Speaker 1: And then they also offered up a space where you
Speaker 1: could type in as much as you wanted about what
Speaker 1: bothered you of those in that search results page. So
Speaker 1: they included that, and that was their measurement of how
Speaker 1: many people detected that there was a bias going on UM.
Speaker 1: And then the results are pretty interesting. So they saw
Speaker 1: again that before they allowed people to do a search
Speaker 1: on the candidates, there was no real there's no contender.
Speaker 1: Nobody cared about these Australian politicians one way or enough.
Speaker 1: To be fair, no one really cared afterwards either, I'm
Speaker 1: just kidding, just kidding. Well, after they were able to
Speaker 1: do the search results, they saw that there was a
Speaker 1: forty eight point four percent increase in the number of
Speaker 1: people who said they would vote for the favored candidate
Speaker 1: of their respective test group. So they called that forty
Speaker 1: eight point four percent the vote manipulation power or VMP
Speaker 1: or VAMP. Yes, I guess to be VOMP if it's
Speaker 1: if it's vote so SOT eight point four percent. That
Speaker 1: sounds like a kind of a significant number to me,
Speaker 1: But keeping in mind these are people who know probably
Speaker 1: nothing about them. We're going to explore that in a
Speaker 1: later part of this experiment, but just in these test conditions,
Speaker 1: that's pretty interesting that people were that willing to be
Speaker 1: swayed just by biased ordering of search results in Google. Again,
Speaker 1: let us state all the same articles were available to everybody.
Speaker 1: It was just a question of in what order they
Speaker 1: were presented. Yeah, And and it turned out that most
Speaker 1: people could not detect that there was a bias going on.
Speaker 1: Of the participants said that they did not detect any
Speaker 1: kind of bias, and they didn't they didn't click that
Speaker 1: something about this bothered me, only clicked on it and
Speaker 1: made some indication that they felt the search engine results
Speaker 1: were not fair in some way. We weren't given the
Speaker 1: full details of all the results in the in the paper,
Speaker 1: but based upon the level of transparency the rest of
Speaker 1: this report has I'm pretty confident that that's that's an
Speaker 1: accurate representation of why happened. Because one of the things
Speaker 1: that Joe and I were talking about this before the podcast.
Speaker 1: One of the things that we were both impressed by
Speaker 1: was that, uh, they were very forthcoming about the limitations
Speaker 1: of their testing procedure and not being an expert, just
Speaker 1: as a lay reader, this seems like a very good
Speaker 1: study to me. It seemed very conscientious and very thoughtful
Speaker 1: and self critical about its own about the limitations of
Speaker 1: what conclusions we could draw from their results. Also, the
Speaker 1: fact that they use double blind made me happy. But
Speaker 1: at any rate, so they repeated this experiment a couple
Speaker 1: of times in San Diego, the experiments one through three,
Speaker 1: we're all in San Diego, and they decided, well, maybe
Speaker 1: we need to bring down that number even lower of
Speaker 1: the number of people who realize that there was manipulation
Speaker 1: at play. So they said, well, how can we do that?
Speaker 1: And they thought, well, one thing we could do is
Speaker 1: we still give the favorable results for candidates to you know,
Speaker 1: groups B and C, group A being the control group.
Speaker 1: But let's insert at the fourth position an article of
Speaker 1: the opposite viewpoint. Okay, so you would get results where
Speaker 1: the first page is all pro Abbot stuff, going eventually
Speaker 1: into the neutral ones in the later pages, and then
Speaker 1: finally getting to the pro Goillard stuff in the final pages.
Speaker 1: But in the third or the fourth or the third place.
Speaker 1: On the first page, there'd be one pro Guillard article
Speaker 1: that was in the first time they ran it for
Speaker 1: experiment too, was in the fourth position. In the third
Speaker 1: experiment third experiment, they moved it to the third position,
Speaker 1: and they found, first of all, the interesting thing was
Speaker 1: if they put it in the fourth position, it actually
Speaker 1: made more people willing to vote for the favored candidate
Speaker 1: a sixty three point three p v MP. That kind
Speaker 1: of makes sense to me because it seems like they'd
Speaker 1: still be getting this overall pretty biased message, but they'd
Speaker 1: start to get the impression based on just one result
Speaker 1: going the other way that yeah, I'm getting the full
Speaker 1: picture here. Yeah, the search results are honest, right exactly,
Speaker 1: because when you see just everything from one from one position,
Speaker 1: then you start going like, is there a bias here?
Speaker 1: Maybe maybe someone over at Google's really pro abbott or
Speaker 1: cadoodle ioodle. So this also meant that fewer people were
Speaker 1: able to tell that something was not on. The up
Speaker 1: and up of the participants were did not indicated all
Speaker 1: that they were aware of a bias, so down to
Speaker 1: of the group. Then they did it a third time,
Speaker 1: and this time that's when they moved it to the
Speaker 1: third position instead of the fourth position, and uh, this
Speaker 1: time it was a lower VMP of thirty six point
Speaker 1: seven percent. But they didn't get any hits at all
Speaker 1: on a detection of bias. So, if your goal is
Speaker 1: to make sure no one knows you're messing around, you
Speaker 1: put that you put the the one negative position in
Speaker 1: in number three, and you'll get a lower result overall,
Speaker 1: but no one knows you're up to something, and it's
Speaker 1: still a significant result. All right, So this is very interesting.
Speaker 1: But obviously the next step is what would happen if
Speaker 1: we tried to scale up these sample sizes and get
Speaker 1: more people to take this test. Yeah, so sample size
Speaker 1: is one of those biases in any study that is problematic. Yeah. More,
Speaker 1: the more people you test, the less chance there is
Speaker 1: of error. Right, with a hundred into you, you'r you
Speaker 1: don't have enough quote unquote random element in there too
Speaker 1: to make sure that you're you haven't skewed the results
Speaker 1: just through the sample size, just choosing a certain demographics
Speaker 1: or something like. Yeah. So so next they went to
Speaker 1: the mechanical Turk. Yeah, mechanical turk, Amazon's mechanical Turk, which
Speaker 1: has been designed to help gather subjects for various types
Speaker 1: of of of studies and they were able to get
Speaker 1: one volunteers. They got volunteers from all the fifty states
Speaker 1: in the United States UH, and the demographics were interesting.
Speaker 1: They weren't necessarily aiming for specific demographics. What it turned
Speaker 1: out to be was that nineteen point five percent of
Speaker 1: the subjects were self identifying as conservative and fifty point
Speaker 1: two percent as liberal UH, and then you had you know,
Speaker 1: others on that spectrum as well, And it was just
Speaker 1: interesting that that was the numbers, especially when you start
Speaker 1: looking at the results per demographic towards the end. So
Speaker 1: they found out that the the vmp UH in this case,
Speaker 1: they repeat the same test, so it's the same Australian
Speaker 1: Prime minister candidates. They found that it was thirty seven
Speaker 1: point one percent, or once you do some post stratification adjustments,
Speaker 1: it's thirty six point seven percent. Post gratification is all about,
Speaker 1: you know, you take the the sample size that you
Speaker 1: had and the demographics that were based off of that,
Speaker 1: and then you have to scale it up to the
Speaker 1: general demographics and adjust the numbers based upon the weight
Speaker 1: of those numbers. So that's why it dipped down to
Speaker 1: thirty six point seven percent. Once they finished with that,
Speaker 1: they the results indicated that some demographics were more vulnerable
Speaker 1: to this manipulation than others. So depending on some facts
Speaker 1: about you, you might be more susceptible to bias ordering
Speaker 1: and information presentation. So one weird example that stuck out
Speaker 1: to me is apparently self labeled divorces were more vulnerable
Speaker 1: than self labeled married subjects. Yeah, if you were a
Speaker 1: self labeled divorce Republican and you were pretty much going
Speaker 1: to be led astray by the search engine. This was
Speaker 1: another strange thing it found is that apparently self labeled
Speaker 1: Republicans were more swayed by the order of presentation than
Speaker 1: self labeled Democrats, like by a bunch of Republicans were
Speaker 1: at fifty four point four percent and self labeled Democrats
Speaker 1: were at thirty seven point seven percent. Moderate Republicans were
Speaker 1: the most vulnerable group out of all of them. Keep
Speaker 1: in mind that they were also a small group because
Speaker 1: the overall number of people who identified as Republican was
Speaker 1: or at least conservative anyway, was nineteen. But yeah, they
Speaker 1: had a VMP of eighty percent. The lowest VMP, weirdly enough,
Speaker 1: was in a particular income bracket, very weird to me.
Speaker 1: At forty thousand to forty nine thousand, those people are
Speaker 1: very Google savy that two point five v MP. So, uh,
Speaker 1: this is one of the things where you know, you
Speaker 1: can't necessarily draw broad conclusions based upon this study, but
Speaker 1: it was one of those things they noticed, and they said, hey,
Speaker 1: this might mean that if you wanted to manipulate a
Speaker 1: you know, an election in some way, you would use
Speaker 1: this kind of information to know who to target the most,
Speaker 1: because it would tell you which ones are you going
Speaker 1: to get the biggest return on investment. Assuming that that
Speaker 1: this manipulation actually results in action, which is a big assumption,
Speaker 1: will get to do a little bit later. So so
Speaker 1: you're going to get way more bang for your buck
Speaker 1: if you do skewed search results to divorced moderate Republicans
Speaker 1: as opposed to married people who make forty And what
Speaker 1: about bias in this section of the study, this is crazy? Um,
Speaker 1: so based upon again, they used the same approach as
Speaker 1: before to say, hey, did anything bother you about the
Speaker 1: search results? They there was a forty five percent v
Speaker 1: MP for people who detected a bias and a thirty
Speaker 1: six point three for people who were unaware of a bias.
Speaker 1: In other words, so hold on. The people who said
Speaker 1: something bothered me about these results, I think they were biased,
Speaker 1: were more affected by the biases that being aware of
Speaker 1: the bias. So counter except, we've been conditioned that the
Speaker 1: first page of search results are the best results for
Speaker 1: your query. So if you're conditioned to the point where
Speaker 1: I know that the thing I want, the information I
Speaker 1: need is on that first page, even if I think
Speaker 1: it's biased, well it's got to be right, it's got
Speaker 1: to be the most relevant because people wouldn't lie to me.
Speaker 1: Why would I go to page two. Yeah. So so
Speaker 1: now now they're just saying that that's that's a possible
Speaker 1: answer for that particular that their interpretation right, They don't
Speaker 1: they don't know for sure that that's the case. And also,
Speaker 1: again using the really vague what bothered you about this
Speaker 1: could mean that there's a little gray area around all
Speaker 1: of this as well. Yeah, okay, So there was one
Speaker 1: more phase of the experiment that they carried out, and
Speaker 1: I thought this was a very interesting next step because
Speaker 1: as soon as I was this part in the study,
Speaker 1: I was like, well, all these are having to do
Speaker 1: with candidates that we don't have preformed opinions about or
Speaker 1: almost nobody in the study had preformed opinions, which obviously
Speaker 1: is not how reality works. Yeah, and when when you're
Speaker 1: when you're actually a voter in an election, you probably
Speaker 1: have some preconceptions and biases going into your research. See that.
Speaker 1: But you have a stake in it. If you're a
Speaker 1: US citizen, you have very little stake. And who is
Speaker 1: Prime Minister of Australia in the grand scheme of things. Yes,
Speaker 1: we're all connected and I love all of you as
Speaker 1: brothers and sisters. However, that being said, I mean it
Speaker 1: might affect the probability of getting future crocodile Dundee sequels. Okay,
Speaker 1: I know how I vote on that, so at any rate.
Speaker 1: Uh So. But what they wanted to do was use
Speaker 1: this in a real world situation so they could determine
Speaker 1: how big of an effect, if any, would there be
Speaker 1: in that instance. And so what they did was they
Speaker 1: went to India. There were these enormous elections. There were
Speaker 1: something like eight hundred million potential registered voters, of which
Speaker 1: six hundred I think sixty million went and voted. I
Speaker 1: read that at the time, it was the largest election
Speaker 1: in human history. Yeah. Yeah, so what they wanted to
Speaker 1: do was run the same experiment, but actually using real candidates.
Speaker 1: In this election, they selected two thousand one voters in
Speaker 1: India who had not yet voted on one of three
Speaker 1: candidates for a specific position. They also, again they compensated them.
Speaker 1: The compensation, by the way, included one interesting option. Uh,
Speaker 1: they would either give it ranged from one to four
Speaker 1: dollars depending upon where you were. But they also gave
Speaker 1: an offer of giving donating a dollar fifty to a
Speaker 1: charity that would feed poor Indian children. So at the
Speaker 1: end of it, around dollars was raised for for kids,
Speaker 1: which was awesome. So it's cool that they did that,
Speaker 1: you know, it was that they were giving back into
Speaker 1: this community. Yeah. One of the things they pointed out, Now,
Speaker 1: in all these cases, I think they wanted to specify
Speaker 1: that they were taking care not to cause any harm
Speaker 1: to the participants or to the democratic process. And in
Speaker 1: the other cases it didn't really matter because it was
Speaker 1: referring to past elections in another country. Yeah, there was
Speaker 1: no way that any of the results would ever have
Speaker 1: any effect on things that had already happened because causality, y'all. Now,
Speaker 1: this is an election that's ongoing in the country where
Speaker 1: the people who were participating had the opportunity to have
Speaker 1: their minds changed, and this could actually affect the election.
Speaker 1: But they did say that it was a small enough
Speaker 1: sample size that they didn't think it was going to
Speaker 1: sway the election. Million is not significant. Uh. And also
Speaker 1: one of the interesting things they pointed out is that
Speaker 1: the biases that might come as a result of this
Speaker 1: study would be balanced out because they're doing it for
Speaker 1: all the candidates. In turns right, they were they were
Speaker 1: equally distributed among each of the candidates because the group
Speaker 1: was subdivided into I assumed four groups this time a
Speaker 1: control group, control, and three one for each of the candidates. Um.
Speaker 1: So again they were using real articles that have been
Speaker 1: written about these actual candidates, and then they had to
Speaker 1: figure out how to rank these in search to favor
Speaker 1: each of the candidates and then randomly distribute them for
Speaker 1: the control group. Yeah. They said they had to actually
Speaker 1: get the help of an Indian like consultant to help
Speaker 1: them determine exactly which ones were most biased towards which candidates. Yeah. Yeah,
Speaker 1: and they had a little bit more of a problem
Speaker 1: and into study trying to work that out. Yeah. In fact,
Speaker 1: there they have numbers for the pre optimization and the
Speaker 1: post optimization of the test, which I'll get into in
Speaker 1: a second. Because of that very thing, you know, they
Speaker 1: had to bring a consultant, and because they said, we
Speaker 1: have a limited understanding of Indian politics, being these researchers
Speaker 1: from the United States, uh and and felt that maybe
Speaker 1: what we were presenting people was not truly the best
Speaker 1: arranged list of search results in order to see if
Speaker 1: the effect is real in a real world setting. So
Speaker 1: that's why they hired the consultant. But that was already
Speaker 1: when they had already started this this experiment, So it
Speaker 1: did affect the numbers. I'll get into that in a second.
Speaker 1: So the overall VMP for the whole experiment was only
Speaker 1: quote unquote ten point six percent. But there are a
Speaker 1: lot of things you have to take into consideration here.
Speaker 1: One is that you didn't have these candidates who were
Speaker 1: complete unknowns to the people in question, right. These these
Speaker 1: were people already had some ideas about these various candidates.
Speaker 1: Some people had very strong opinions about these candidates going
Speaker 1: into the experiment, and they found that those people were
Speaker 1: the least likely to be swayed. Yes, So yeah, that
Speaker 1: ten point six percent, that's kind of what you get
Speaker 1: after you look at the pre and post optimization of
Speaker 1: the experiment. So before they brought the consultant on, uh,
Speaker 1: it was trending towards nine point five so lower. Then
Speaker 1: after they got the consultant it went to twelve point
Speaker 1: three percent. So possibly had they brought the consultant on
Speaker 1: from the very beginning, that number would be higher. One
Speaker 1: thing that did impact the number was that there was
Speaker 1: a group that had a very strong counter reaction to
Speaker 1: the search results. Yeah, this was something I don't think
Speaker 1: was encountered in any other phase of this research as
Speaker 1: far as I noticed. But it was a negative VMP,
Speaker 1: meaning that that if you presented search results biased toward
Speaker 1: one candidate, it actually worked again to that candidate, people
Speaker 1: were more likely to vote for one of the other candidates.
Speaker 1: And that one demographic was conservative female voters in India
Speaker 1: with a negative VMP of negative eleven point eight per cent.
Speaker 1: So with this one particular demographic in this test group,
Speaker 1: if you showed them a list of articles that were
Speaker 1: all biased in favor of one politician, it worked against
Speaker 1: that politician's interests, right, and the researchers said, it might
Speaker 1: suggest an oppositional attitude, or it may be a tendency
Speaker 1: to favor and underdog someone that you want to see
Speaker 1: the person who is trailing behind to come from that
Speaker 1: position and take it all. And UH, they said that
Speaker 1: if you eliminated that group from the results, it would
Speaker 1: raise the VMP from ten point six percent to nineteen
Speaker 1: point eight percent. It actually goes up to in the
Speaker 1: twenties depending upon the various UH implications they talked about later,
Speaker 1: but up to nineteen point eight percent. And they said,
Speaker 1: eliminating them from the results is actually not an unfair
Speaker 1: thing to do, because if in a real world setting
Speaker 1: you were attempting to manipulate the results of an election,
Speaker 1: you would specifically target the people that you felt you
Speaker 1: could you could influence, and you would specifically the people.
Speaker 1: In fact, you could even take advantage of this, like
Speaker 1: if you knew this was the effect for certain demographics
Speaker 1: of people, you could pick those people and show them biases.
Speaker 1: You could show them results with bias in the opposite
Speaker 1: direction or at least of the candidate you would least want. Yeah,
Speaker 1: so you could even have it work for you in
Speaker 1: that sense. So when you get that through all of
Speaker 1: these different experiments, was is ultimately mean you got you
Speaker 1: have to get to the true analysis, you know, does
Speaker 1: this actually matter? Well, according to the researchers, they think, yeah,
Speaker 1: they have got a statistically significant and very interesting and
Speaker 1: perhaps worrying set of results on their hands. So I
Speaker 1: want to read this one quote from the from the
Speaker 1: final analysis section of their paper. They said, our investigation
Speaker 1: suggests that with optimized targeted rankings, a VMP of at
Speaker 1: least twenty percent should be relatively easy to achieve in
Speaker 1: real elections. Even if only sixty percent of the population
Speaker 1: had internet access and only ten percent of voters were undecided,
Speaker 1: that would still allow control of elections with win margins
Speaker 1: up to one point two percent. And that includes a
Speaker 1: lot of elections. They're close elections all the time. So
Speaker 1: if you're talking about a very like razor thin kind
Speaker 1: of lead, then something like this would be enough to
Speaker 1: put one candidate in front of the other. Yeah. Certainly
Speaker 1: here in the United States, especially in larger elections like
Speaker 1: the presidential election, that is the case. Yeah, it could
Speaker 1: not that not. I mean, if you look at the
Speaker 1: overall numbers of the presidential election, it may not look
Speaker 1: very close. But you have to remember that the elections
Speaker 1: in the US for presidents are decided state by state, right,
Speaker 1: So what you could do then is target one or
Speaker 1: two key close races in swing states, and those are
Speaker 1: the only things you'd have to push over the edge. Yeah,
Speaker 1: because you already know that tackling any state that's really
Speaker 1: entrenched in one camp or the other is kind of Uh,
Speaker 1: it's a lost cause, there's no point in it. You're
Speaker 1: not going to be able to create enough of a
Speaker 1: swing in that to make a big difference. So if
Speaker 1: you can convert one point two percent of Ohio or
Speaker 1: Florida or whatever state you know is the big swing
Speaker 1: state that year, that can win an election. So uh,
Speaker 1: mostly it looks like these these sort of tactics would
Speaker 1: have would mostly affect undecided voters, people who had not
Speaker 1: already made a decision on on one candidate versus another.
Speaker 1: If you have already made that decision and maybe that
Speaker 1: the search results aren't compelling enough for you to change
Speaker 1: your mind. It's very difficult to change someone's mind anyway,
Speaker 1: especially by resenting them evidence, especially in the realm of
Speaker 1: pologists in harder Yeah, but yeah, if it's science kids.
Speaker 1: But yeah, if the top search results are positive for
Speaker 1: a candidate to an undecided voter, that voter maybe feeling
Speaker 1: more inclined to vote for that candidate. But they do
Speaker 1: also point out, and this is one of those things
Speaker 1: we were mentioning earlier about how they were quick to
Speaker 1: say the limitations of their study. They say that there
Speaker 1: is a known laboratory effect in general, not just for
Speaker 1: this study, but for lots of stuff, a laboratory effect
Speaker 1: where you might observe something in the lab that seems
Speaker 1: really relevant, but in the real world it becomes less so.
Speaker 1: And they said that it may be that this this
Speaker 1: influence is very uh, it's tenuous. It doesn't last very
Speaker 1: long either, So it may be that within an hour
Speaker 1: whatever effect there was kind of wears off. So unless
Speaker 1: you unless you send up the search results page and
Speaker 1: then boot that person out the door to go to
Speaker 1: the polls, it may end up not being a big
Speaker 1: enough effect to actually create action. So but still could
Speaker 1: be it could be the element that that does push
Speaker 1: someone to choose one candidate over another, And that alone
Speaker 1: raises the question of how do we how do we
Speaker 1: account for this, how do we make sure we're aware
Speaker 1: of it, And is anyone doing this on purpose, or
Speaker 1: if they're not even doing it on purpose, what do
Speaker 1: we do to make sure it doesn't like affect things. Well,
Speaker 1: that's the really crazy thing to me is that even
Speaker 1: if even if no one is purposefully manipulating Google search
Speaker 1: results like this, the algorithm could be doing it on
Speaker 1: its own. Oh yeah, yeah, I mean there could be
Speaker 1: an effective bias coming out of Google results that is
Speaker 1: already changing the outcomes of elections without anybody wanting it
Speaker 1: to happen. It could just be for it to happen
Speaker 1: at any rate. Yeah, it could be an accidental byproduct
Speaker 1: of something that the algorithm does for completely neutral political
Speaker 1: not political reasons. Well, and you also have to remember
Speaker 1: that algorithms are designed by people, not designed. They're not
Speaker 1: spawned by you know, deep thought, the uncaring, the uncaring
Speaker 1: thinking machine at the heart of the universe. It's actually
Speaker 1: stuff that the human beings have designed, and sometimes that
Speaker 1: introduces a bias. And in fact, I would argue that
Speaker 1: the concept of page ranking already has at least some
Speaker 1: element of bias. There's no real objective way of saying
Speaker 1: this page is more important than this page. Therefore, this
Speaker 1: this first page needs to be ranked first in the
Speaker 1: search results. Eventually you have to come to uh decision
Speaker 1: that may have some bias to it to to actually
Speaker 1: make that determination. Now, that bias may not be inherent
Speaker 1: in the algorithm. It may be inherent in how everyone
Speaker 1: else is treating that page, and that's how I got
Speaker 1: to number one. But the algorithm is the thing that
Speaker 1: determines what criteria are most important when making that determinations,
Speaker 1: It's it's a long winding road to get there. But
Speaker 1: the ultimate the ultimate answer here is that search results
Speaker 1: could totally like effect an election. Y'all. Okay, to be fair,
Speaker 1: people aren't getting all of their information about candidates for
Speaker 1: elections from Google. Right. Sometimes they watch a man on
Speaker 1: TV yell at them. Hey, that's my favorite thing to do.
Speaker 1: Sometimes they'll change the channel and watch a different man
Speaker 1: on yell at them. Or sometimes they listen to their
Speaker 1: friends on Facebook who are ranting about it. Right, yeah, yeah,
Speaker 1: So you know, there are so many different avenues that
Speaker 1: are open to us about, you know, where we get
Speaker 1: our information for this sort of stuff. Um, in a
Speaker 1: world where we would only get this information from the internet, obviously,
Speaker 1: this would have a much greater effect than it does already.
Speaker 1: So you know, you can ask the question of how
Speaker 1: many people actually use the Internet to do active research
Speaker 1: on political candidates, and that would give you a better
Speaker 1: idea of how effective this is in the long run.
Speaker 1: I honestly don't know that answer. You know, I don't
Speaker 1: know how many people of the of the people who
Speaker 1: go and vote, or the people who are considering voting,
Speaker 1: how many of them take the time to actually do
Speaker 1: Google style research on candidates as opposed to just seeing
Speaker 1: what their friends are saying or uh, consuming stuff in
Speaker 1: other forms of media. Well, of course, Google results aren't
Speaker 1: the only venue through which Internet. Internet companies of various
Speaker 1: kinds could change the outcomes of elections without even producing
Speaker 1: content of their own, just by choosing in what order
Speaker 1: and when we see content. To think about Facebook again,
Speaker 1: you know, we talked about that get out the vote
Speaker 1: message earlier on Facebook. What if they just showed it
Speaker 1: to one party, or what if the things that were
Speaker 1: featured in your news feed the algorithm for selecting those
Speaker 1: things had a political slant. Yeah, So, like, like, let's
Speaker 1: say that my good buddy uh posts a a um
Speaker 1: An article about Pinchio hula Han, which normally I would
Speaker 1: like the heck out of But because Facebook is anti pinch,
Speaker 1: Facebook is now in the anarchist crab camp because they've
Speaker 1: seen the light of I always forget clause mc grabby clause,
Speaker 1: so you don't even know your candidates name. How am
Speaker 1: I supposed to dr mc grabby clause supports freedom? Now
Speaker 1: doctor huh is a PhD? I assume, And anyway, pincio
Speaker 1: hula hand. Because Facebook hates pincio hula hand, they end
Speaker 1: up essentially burying that that link. Yeah, so on mind feed,
Speaker 1: which features stuff that people have posted, it doesn't pop
Speaker 1: up at all. If I were to go to my
Speaker 1: friends feed, I would see it there because Facebook allows
Speaker 1: you to go ahead and post it to your own feed.
Speaker 1: They just don't feature that anywhere else. So this is
Speaker 1: the thing. You might not have even realized this, Like,
Speaker 1: you don't see everything your friends on Facebook post unless
Speaker 1: you've specifically opted to follow them directly. There there are
Speaker 1: a lot of things and placed a certain amount of
Speaker 1: importance on what they post by liking and commenting on it,
Speaker 1: or or by opting in via a button that is
Speaker 1: ridiculously hidden, like the close friends button. If you do
Speaker 1: a close friends button only do you get everything. It
Speaker 1: notifies you when someone posts something. Never ever make me
Speaker 1: your close friend. I post too much to Facebook. You
Speaker 1: don't want that to happen to you. I'm just saying
Speaker 1: I I don't want that to happen to you either.
Speaker 1: Thank you for your concern, Jonathan, You're welcome. But so so,
Speaker 1: this kind of sorting is happening all the time on
Speaker 1: Facebook based on what posts you interact with and and
Speaker 1: your own biases. You what we see on Facebook is
Speaker 1: determined by what we interact with and and so therefore,
Speaker 1: you know, aside from the from the echo chamber that
Speaker 1: we all create by choosing who were friends with on Facebook,
Speaker 1: and you know, under the assumption that most of your
Speaker 1: friends are probably like minded, uh that it just it
Speaker 1: becomes even more smaller and echo ere when you take
Speaker 1: these kind of algorithms into account. Yea. So it's one
Speaker 1: of those things where, uh, you know, it would be
Speaker 1: entirely possible for a company like Facebook or Google to
Speaker 1: manipulate things so that, uh, a specific side of the
Speaker 1: story is being favored over another, and uh and that
Speaker 1: could definitely affect how we perceive those stories. Or I
Speaker 1: just want to say one more thing. It could be
Speaker 1: not just what we see and what we don't see,
Speaker 1: but the order in which we see it in you feed,
Speaker 1: because we know, as we've said before, things like recency
Speaker 1: and priority matter. Yeah, yeah, exactly, the most recent thing
Speaker 1: you you have encountered is more likely to have a
Speaker 1: longer effect on the first thing you encounter, the first
Speaker 1: thing on a list. Yeah. Boy, I never want to
Speaker 1: be the person who has to make the decision of
Speaker 1: if you're making a ballot, whose name comes first. That's
Speaker 1: you know. You could like, well, we're just gonna go
Speaker 1: alphabetical order and let let and leave the blame to
Speaker 1: the Latin alphabet. Seems like that. I don't know how
Speaker 1: they actually do it. Seems like they could randomize that
Speaker 1: for equality, right, they could randomize it, especially with electronic
Speaker 1: you can you can make it so that every single
Speaker 1: person who comes up gets a random or range. Yeah,
Speaker 1: or however many candidates there are. But in the United States,
Speaker 1: come on, let's be honest, this is pretty much what
Speaker 1: we tend to get. The crab party is on the rise. Yeah,
Speaker 1: the lobster imperialists really don't care how the voting goes.
Speaker 1: That's kind of how imperialist. Are I like to imagine
Speaker 1: that there's a vote vote algae you guys, because you
Speaker 1: can really you can suffoc so on that happy note. Uh,
Speaker 1: this was really an interesting thing to look at. The
Speaker 1: study like, like we said, is first of all, it's
Speaker 1: incredibly accessible. It's very easy to read. It is not
Speaker 1: like for open access. Yes, definitely check that out if
Speaker 1: you're interested in finding out exactly how they went about
Speaker 1: putting this this uh, this study together and the criteria
Speaker 1: they used. It is really easy to read. It's fascinating
Speaker 1: and like I said, you know, it's I admire their
Speaker 1: work and their approach to their work that I think
Speaker 1: it's something that I want to see more of in
Speaker 1: all sorts of areas of science. And meanwhile, if you
Speaker 1: guys have suggestions of future topics that we can tackle
Speaker 1: here on Forward Thinking, or you've got anything you want
Speaker 1: to say about this episode, send us a message. The
Speaker 1: email addresses f W Thinking at how Stuff Works dot com,
Speaker 1: or drop us a line on Twitter we are FW
Speaker 1: thinking there, or just search f W thinking over on Facebook.
Speaker 1: We'll pop right up. You can leave us a message
Speaker 1: and we'll salve to you again really soon for more
Speaker 1: on this topic in the future of technology, I'll visit
Speaker 1: forward thinking dot Com, brought to you by Toyota. Let's
Speaker 1: Go Places