Speaker 1: Welcome to Stuff to Blow your Mind from how Stuff
Speaker 1: Works dot com. Hey, welcome to Stuff to Blow your Mind.
Speaker 1: My name is Robert Lamb, and I'm Julie Douglas. You know, Julie.
Speaker 1: Science has done some good things for us. It's done
Speaker 1: a lot of good things. I mean, it has really
Speaker 1: forwarded humanity right and made us, in some ways the
Speaker 1: kind of success story of the species that we are. Yeah,
Speaker 1: it's kind of the skeleton of human culture, that the
Speaker 1: thing upon which we grow and continue to grow. And
Speaker 1: you know, and you can look at just about any
Speaker 1: area right, medical science, exploration of inner and outer space, um,
Speaker 1: increasing knowledge of the self, the brain, the connection, our
Speaker 1: connection to the from the brain to the body. I mean,
Speaker 1: pretty much everything we talk about every week is is
Speaker 1: a testament to what science is doing and has done
Speaker 1: for humans. And while the pursuit of science, what we
Speaker 1: think about as science, has been around for a very
Speaker 1: long time, this pursuit of knowledge and truth, the word
Speaker 1: scientists is only one hundred and eighty years old. Before that,
Speaker 1: a person might be called a natural philosopher. And before
Speaker 1: that you had economists, you had philosophers, and what we
Speaker 1: now call scientists. All co mingling under the same roof,
Speaker 1: and this affected how science and and how we think
Speaker 1: of it was defined and pursued. And we sort of
Speaker 1: assumed that science and the scientific method were in place
Speaker 1: from the get go, but in fact they hadn't really
Speaker 1: been defined in the rules tightened, uh, you know, until
Speaker 1: a couple of hundred years ago, because economists were pulling
Speaker 1: for deductive reasoning, right, and scientists we're saying, no, I
Speaker 1: think there's there's more of this inductive reasoning, which is
Speaker 1: this premise that you you take an idea and then
Speaker 1: you try to take it down to the studs and
Speaker 1: prove it wrong, even theo you might want it to
Speaker 1: be right. And the whole idea there is that you're
Speaker 1: trying to get at this kind of truth. And this
Speaker 1: is now something called the scientific method. But we sort
Speaker 1: of take this for granted, this this fact that this
Speaker 1: is only a fairly recent development in the long history
Speaker 1: of humans. Yeah, I mean, it's it's basically how science works.
Speaker 1: There were people who managed to make it work in
Speaker 1: the past, but it was until recently that we actually
Speaker 1: said this is what works, and this is why we
Speaker 1: should stick to. Now. A lot of the advances in
Speaker 1: century you can, you can boil down to a simple
Speaker 1: idea trust, but very verify. And this plays into our
Speaker 1: peer view system. Right, one scientist writes a paper, or
Speaker 1: a team of scientists write a paper, maybe there's a
Speaker 1: big breakthrough in it, maybe not. But then the idea
Speaker 1: is that their peers come along, look at the paper
Speaker 1: and try to replicate the results, just you know, tear
Speaker 1: it apart, see what's happening in the paper and say, yes,
Speaker 1: I agree this is working, or I have problems with
Speaker 1: this or that, or this is complete bunk. Yeah. I mean,
Speaker 1: it's this idea that science can police itself. And yet
Speaker 1: we have some statistics coming out that point to other
Speaker 1: factors going on and that perhaps we're not pursuing knowledge
Speaker 1: for knowledge itself in some cases or truth and and
Speaker 1: we'll discuss more of those factors in a bit. According
Speaker 1: to the Economists article How Science Goes Wrong, in two
Speaker 1: thousand twelve, biotech firm am Jin reported that they could
Speaker 1: reproduce just six of fifty three landmark studies in cancer research,
Speaker 1: and earlier Bear the drug company, managed to repeat just
Speaker 1: a quarter of sixty seven similarly important papers. Now we're
Speaker 1: not taking on this topic today because we think that
Speaker 1: we are experts on this topic, but by no stretch
Speaker 1: of imagination are we. But we do rely on a
Speaker 1: lot of studies, and so we wanted to point this
Speaker 1: out today to just for ourselves better understand what are
Speaker 1: the conditions that lead to a good, solid study or experiment.
Speaker 1: What are the conditions that lead to dubious data? Yeah,
Speaker 1: certainly worth keeping in mind too when you find yourself
Speaker 1: reading science journalism articles, you know, uh that asking yourself, well,
Speaker 1: what is the study? You know, what are there problems
Speaker 1: and it what could the problems be? Because of what
Speaker 1: we'll discuss, there are a number of problems that can
Speaker 1: and do occur in modern peer of viewed science. Now,
Speaker 1: one of the things that will come up sometimes when
Speaker 1: people write on this topic is careerism as one of
Speaker 1: the factors that is problematic. And that's because we've all
Speaker 1: heard the maximum published or parish right, and the spirit
Speaker 1: of it is not so bad. I mean, the spirit
Speaker 1: of it is really like less than a threat and
Speaker 1: more like, hey, this is a challenge to push science forward.
Speaker 1: Put forth your multiple lines of evidence, your hypotheses, your theories,
Speaker 1: because we all want to share information. We wanted to
Speaker 1: it apart, we want to try to validate it or
Speaker 1: invalidate it, and generally create a better understanding of the
Speaker 1: topic or the issue. So again it's an attempt at
Speaker 1: reaching some sort of truth. And yet the reality of
Speaker 1: published or perish now is more that it's this kind
Speaker 1: of pressure to produce. So it's not enough for say
Speaker 1: a faculty member at university to write a few really
Speaker 1: good papers a year. Now they have this pressure to
Speaker 1: write several And so there's this idea that questionable results
Speaker 1: could come out of this, and instead of maybe making
Speaker 1: it to a first tier journal, maybe that data goes
Speaker 1: to a third tier journal. And yet it shouldn't necessarily
Speaker 1: go any place. And the problem, as outlined in the
Speaker 1: Economist article how Science Goes Wrong, is quote. In order
Speaker 1: to safeguard their exclusivity, the leading journals impose high rejection
Speaker 1: rates in excess of submitted manuscripts. The most striking findings
Speaker 1: have the greatest chance of making it onto the page.
Speaker 1: Little wonder that one in three researchers knows of a
Speaker 1: colleague who has pepped up a paper by say, excluding
Speaker 1: inconvenient data from results based on a gut feeling. So
Speaker 1: we're talking about here is cherry picking information. And then
Speaker 1: all of this, this kind of careerism is compounded by
Speaker 1: the pressure to generate grant funding. So there's this idea
Speaker 1: that more and more scientists are having a bigger percentage
Speaker 1: of their salary covered by contingent or research funding dollars.
Speaker 1: So that means that you now have this pressure to
Speaker 1: keep the flow of funding going with positive results. So
Speaker 1: you can say, yeah, see, this is exactly what I
Speaker 1: thought was going to happen, proving out um. That shouldn't
Speaker 1: be the case. There shouldn't be those sort of strings
Speaker 1: tied to it, and in an ideal world that wouldn't
Speaker 1: be the case. But that's what we're dealing with. And
Speaker 1: then there's this failures to prove hypothesis are actually rarely
Speaker 1: offered for publication, let alone accepted. Uh, you know, and
Speaker 1: you can sort of squirrel away a lot of this
Speaker 1: to you know, what scientific journal doesn't want to be
Speaker 1: on the forefront of science, you know, full of amazing
Speaker 1: new discoveries and and and wonderful new ideas, right that's
Speaker 1: you know, that's really essential to the overall drive of science.
Speaker 1: You don't want to fill your your paper with a
Speaker 1: bunch of failures, right, But the failures are important, right.
Speaker 1: You need to know what hasn't worked so you can
Speaker 1: try and figure out what does work. You need to
Speaker 1: know what's false so you can figure out what's true. Yet,
Speaker 1: in two thousand thirteen, negative results are accounted for only
Speaker 1: fourteen percent of published papers, and that was down from
Speaker 1: thirty in nineteen ninety. And then in a similar vein
Speaker 1: we see the peer of view process UM often sees
Speaker 1: peers missing the errors in the paper. The very thing
Speaker 1: they're supposed to do is, you know, figure out what's
Speaker 1: what's potentially wrong with this work. So both of these
Speaker 1: into handicap the process to a certain extent. You know,
Speaker 1: it's interesting because my daughter's school has nine different design
Speaker 1: principles of education, and there this is something that they
Speaker 1: actually present to the student. So kindergarten reserve being taught
Speaker 1: about failure and actually celebrating failure for this very reason,
Speaker 1: because the idea, again is that you cannot have successes
Speaker 1: without failures. And uh makes me think about Edison and
Speaker 1: the light bulb and the hundred plus iterations of the
Speaker 1: light bulb, all the failures that proceeded those, And yet
Speaker 1: that's not the flashy stuff, right, that's not what necessarily
Speaker 1: a first tier journal is going after. Like, hey, tell
Speaker 1: me about your spectacular failure. Yeah, Like I keep thinking
Speaker 1: about science in terms of slime mold. We did an
Speaker 1: episode on the slime mold way back, where you would
Speaker 1: put a slime mold in a maze, and it's solving
Speaker 1: the maze to get to resources on the outside of
Speaker 1: the maze. And so these tendrils of slime mold or
Speaker 1: trailing through the mazing if they reach a dead end,
Speaker 1: and that tendril dies and fades back and it doesn't
Speaker 1: go down that way again. And science kind of works
Speaker 1: the same way that you need to know which where
Speaker 1: the dead ends are, otherwise you're just gonna keep sending
Speaker 1: your tendils down there. Well. And then it's also this
Speaker 1: such an elegant analogy because they're going after that sugar, right,
Speaker 1: that that resource, and so they're eventually going to find
Speaker 1: themselves to the success story of the resource. But then
Speaker 1: it becomes this question of is that resource that piece
Speaker 1: of sugar that the slime bowl is after. Is this
Speaker 1: truth or is this money? And we'll talk a little
Speaker 1: bit more about that later, but I thought at this
Speaker 1: point I would go ahead and drop in a little
Speaker 1: information about over generalization and extrapolation of results, because this
Speaker 1: can occur in two ways. The first is applying findings
Speaker 1: from one target group to another target group within the
Speaker 1: same population. So an example would be you have this
Speaker 1: new cholesterol drug and it's been tested on females age
Speaker 1: uh it is thirty two fifty. Well, you can't make
Speaker 1: the assumption that the drug can also do the same
Speaker 1: thing for a different population, say women over sixty five
Speaker 1: or men. The second fallacy is applying the survey results
Speaker 1: to population is not living in the area in the survey.
Speaker 1: So this is this to me was very clear cut.
Speaker 1: Let's say that you're trying to establish the mortality rate
Speaker 1: for a certain neighborhood within a zip code. All right,
Speaker 1: you do the research, you do the surveying, and then
Speaker 1: you you've got your data. Now it would be beneficial
Speaker 1: to find out what other neighborhoods mortality rate. But you
Speaker 1: make the assumption that just because the borders of this
Speaker 1: other neighborhood are butting up against the one that you've
Speaker 1: just surveyed, that they have the same mortality rate. Well,
Speaker 1: that is erroneous thinking, because as we know and we
Speaker 1: have seen over and over again you can have really
Speaker 1: poor neighborhoods betting up against very prosperous ones and that
Speaker 1: excuse the data because the very protuluh ones are going
Speaker 1: to have a far different mortality rate than a poor neighborhood.
Speaker 1: And yet these are some of the things that lead
Speaker 1: can with studies and experiments. And then of course there
Speaker 1: is conflict of interest, which is a big one. UH.
Speaker 1: And we can date a lot of this back to
Speaker 1: the Bio Dule Act of nineteen eighty and this came
Speaker 1: along to encourage technology transfer from universities to industry. The
Speaker 1: IDBA being that it would facilitate financial relationships between academic
Speaker 1: biomedical researchers and the biotechnology industry. And you know, obviously
Speaker 1: there's a lot of good that was going to come
Speaker 1: out of this and has come out of this. Uh.
Speaker 1: They leave these uh. These relationships lead to the development
Speaker 1: of improved drugs and medical devices. UH. But on the
Speaker 1: other hand, there's this huge financial aspect of the relationship.
Speaker 1: Of financial relationship emerges relationships that can cause conflicts of
Speaker 1: interest between a researchers scientific and ethical principles and that
Speaker 1: gleam of financial gain coming background to what you said
Speaker 1: about what is the what is the bait on the
Speaker 1: outside of the maze? Is it? Is it knowledge and understanding?
Speaker 1: Is it? Is it increasing our scientific understanding of a
Speaker 1: particular ailment? Or is it mere financial gain? And of course,
Speaker 1: financial gain for a biomedical corporation tends to boil down
Speaker 1: to treatment, the drugs that can be thrown at a
Speaker 1: particular ailment, the medical devices that can be thrown at
Speaker 1: a particular ailment. And in a two thousand nine study
Speaker 1: from Dr Rueschmajas, Assistant Professor of Radiation Oncology at the
Speaker 1: University of Michigan Medical School, compared a thousand, five thirty
Speaker 1: four studies involving cancer research, found that studies with with
Speaker 1: industry funding focused on treatment again drugs, medical devices sixty
Speaker 1: of the time compared to thirty six percent of the
Speaker 1: time for other studies not funded by industry, and the
Speaker 1: studies funded by industry focused on epidemiology, prevention, risk factors,
Speaker 1: screening and other diagnostic methods only twenty percent of the
Speaker 1: time versus forty seven for studies with no declared industry funding.
Speaker 1: So the take home here seems to be the more
Speaker 1: money is involved from these from the biotech industry. The
Speaker 1: more focus there is going to be on the mere
Speaker 1: treatment of an ailment versus uh um, you know, actually
Speaker 1: being able to prevent it or figure out how to
Speaker 1: screen it through looking at risk factors, which my lead
Speaker 1: to misleading statistics or interpretation about the data. And what
Speaker 1: I'm talking about is absolute versus relative percentages. This is
Speaker 1: from the article bad Science, Common Problems and research Articles.
Speaker 1: This was published on Health Readings. Quote supposed that there
Speaker 1: was a medical problem that caused two people in one
Speaker 1: million to have a stroke, and suppose there was a
Speaker 1: treatment that would reduce the problem to only one person
Speaker 1: in one million. This would be an improvement of point
Speaker 1: zero zero zero one percent in an absolute sense, or
Speaker 1: or as this author says, no big deal, right. However,
Speaker 1: if it had been hoarded using relative percentages, it could
Speaker 1: have been stated quote new medical treatment yields a rediction
Speaker 1: and reduction and risk of stroke, and this would be
Speaker 1: very misleading. But it's unfortunately a common practice that you
Speaker 1: see from time to time, and so again you see
Speaker 1: how that's it's not exactly wrong. It is a fifty
Speaker 1: percent reduction in the two and one million people, but
Speaker 1: it's not really accurate saying. It's just how how do
Speaker 1: you end up using? How the effect the overall statistics
Speaker 1: that you're dealing with. Yeah, semantics matter. Now. Another area
Speaker 1: of concern is that of unpublished clinical trials. A two
Speaker 1: thousand twelve study from Yale School of Medicine researchers found
Speaker 1: that fewer than half of a sample of trials primarily
Speaker 1: or partially funded by the National Institutes of Health were
Speaker 1: published within thirty months of completing the clinical trial. So,
Speaker 1: in other words, the research refindings here are not being
Speaker 1: emanated half the time, So the scientific process is disrupted,
Speaker 1: undermining the effort and the available material for peer of view. Now,
Speaker 1: according to study author Dr Joseph Ross, they're probably a
Speaker 1: number of reasons for lack of publication, such as not
Speaker 1: getting accepted by a journal and we already hit on
Speaker 1: the high rejection rates, or not prioritizing the dissemination of
Speaker 1: research findings in the study. Either way, this disrupts the process.
Speaker 1: This disrupts the strengths of the peer of view system.
Speaker 1: Another factor is something called selective observation. Now, you've probably
Speaker 1: experienced your own selective observation before. My example is every
Speaker 1: time I get into the shower of my phone rings, right. Uh,
Speaker 1: and it's a perception that is based on the annoyance
Speaker 1: of my phone ringing and my inability to get to it.
Speaker 1: But then I, you know, I tended to disregard all
Speaker 1: the times that my phone didn't ring well as in
Speaker 1: the shower, and so I was practicing confirmation biased and
Speaker 1: oring the other data, skewing my own statistics. So selective
Speaker 1: observation in science is essentially trying to land on a
Speaker 1: conclusion based on an existing bias or belief. For example,
Speaker 1: a researcher who studying obesity may have a bias that
Speaker 1: obese people lack will power, and as a result, they
Speaker 1: may construct an experiment that involved a plate of donuts
Speaker 1: and a conference froom work. But if that researcher only
Speaker 1: records data about ABS subjects and doesn't record non ABS subjects, well,
Speaker 1: then they have a biased experiment on their hands. In
Speaker 1: other words, Uh, if they don't go out of their
Speaker 1: way to try to prove themselves wrong, they're not exercising
Speaker 1: the principles of scientific method. All right, you know, let's
Speaker 1: take a quick break and when we come back we
Speaker 1: will discuss weird science. All right, we're back. Weird science,
Speaker 1: weird psychology, and I'm not talking about the eighties classic
Speaker 1: as it is. Um No, weird is a phenomenon that
Speaker 1: plagues a lot of psychology and other social science studies.
Speaker 1: This is when the Protestants are overwhelmingly This is where
Speaker 1: weird comes in Western E for educated, and they're from
Speaker 1: I for industrialized, ARE for rich, and D for democratic countries.
Speaker 1: So weird humans are serving as the basic test subjects
Speaker 1: in a lot of these studies. And you can also
Speaker 1: add in that weird humans are also often college students
Speaker 1: in the United States, participating in studies for class credit. So,
Speaker 1: especially in the social sciences, the risk is that so
Speaker 1: called weird populations are actually the outliers of human population
Speaker 1: as opposed to a good standard example of human behavior.
Speaker 1: And you know, you see this, You see shades of
Speaker 1: this time and time again. Right. You look at a
Speaker 1: study and it was clearly a study that was conducted
Speaker 1: on campus with students, and in your better studies you
Speaker 1: see them branching out from that and saying, uh um, well,
Speaker 1: all right, when this first study we looked at students,
Speaker 1: but then we went into an impoverished neighborhood or in
Speaker 1: some cases, then we looked at some US participants. Then
Speaker 1: when we also went and looked at some participants in
Speaker 1: Hong Kong, that sort of thing. Um. And so obviously
Speaker 1: there are a lot there's a lot to consider here
Speaker 1: with the software of psychology, right, because there's so much
Speaker 1: about human culture and uh and in your relations within
Speaker 1: your particular group. But it also bleeds into the hardware
Speaker 1: of physiology. In two thousand fourteen, Liverpool University had a
Speaker 1: study examining rapid eye movements called cicades among groups of
Speaker 1: mainland Chinese, British Chinese, and white British test subjects, and
Speaker 1: he found that Chinese ethnicity was more of a factor
Speaker 1: than culture in high cicade counts. So the mainland Chinese
Speaker 1: groups scored high cicade numbers as did the British Chinese counterparts,
Speaker 1: despite the many cultural differences between the two groups. So
Speaker 1: lead author Dr Paul Knox argued, quote, the human brain
Speaker 1: is not just amazingly complex in general, but also highly
Speaker 1: variable across the human population. Mm hmm. And that variability
Speaker 1: takes us to the next entry here, which is animals. Now,
Speaker 1: we have talked about how much rodents have um contributed
Speaker 1: to science, and they absolutely have, but we do have
Speaker 1: problems where animal studies do not reliably predict human outcomes.
Speaker 1: And this topic is really a complex one, but there's
Speaker 1: a paper on the topic by Michael B. Bracken who's
Speaker 1: from Yale University, and he writes in his paper why
Speaker 1: animal studies are often poor predictors of human reactions to
Speaker 1: exposure that one reason is probably because animal experiments do
Speaker 1: not translate into replications, and human trials are into cancer
Speaker 1: chemoprevention because as they're poorly designed, conducted, and analyzed. Now,
Speaker 1: another possible contribution to failure to replicate the results of
Speaker 1: animal research and humans is that reviews and summaries of
Speaker 1: evidence from animal research are inadequate when it comes to methodology,
Speaker 1: and one survey, only one in ten thousand Meadline records
Speaker 1: of animal studies were tagged as being meta analysis, is
Speaker 1: compared to one and one thousand human studies, and in
Speaker 1: recent reports, the poor quality of research was documented by
Speaker 1: a comprehensive search of Medline, which found only twenty five
Speaker 1: systematic reviews of animal research. Other studies similarly found only
Speaker 1: thirty and fifty seven systematic reviews of any type of
Speaker 1: animal researcher so Um. The reason that Bracken points us
Speaker 1: out is because he says these kind of deficiencies are
Speaker 1: important because animal research often provides the rationale for hypotheses
Speaker 1: studied by epidemiologists and clinical researchers. Moreover, if you look
Speaker 1: at the genetics of this, it gets even more muddled.
Speaker 1: And the reason for that is because with rodents, and
Speaker 1: one of the reasons why we use them is because
Speaker 1: we can change their genetic background within a couple of generations.
Speaker 1: We can tinker with the genes. And that's great because
Speaker 1: that can really help us to study certain conditions. However, Um,
Speaker 1: those rodents would yield really consistent results and disease expression.
Speaker 1: But humans, we are far more wild West when it
Speaker 1: comes to genetics and the genetic background, and that would
Speaker 1: factor in how the human disease is expressed, and this
Speaker 1: would yield a mismatching results between humans and animals. It's
Speaker 1: a layer cake of animal confusion. Indeed, it is um. Now,
Speaker 1: on top of everything we've discussed here, there are plenty
Speaker 1: of additional methodological pitfalls, and we're we're gonna include a
Speaker 1: link on the landing page of this episode to a
Speaker 1: fabulous page that has a list of about sixty of them,
Speaker 1: and we're not going to go into all into detail
Speaker 1: on all of them here, but just to give you
Speaker 1: an example, this includes the likes of the cebo effect,
Speaker 1: which we've discussed at length before, and in which the
Speaker 1: the individual receiving the sugar pill ends up actually getting
Speaker 1: some sort of biological benefit from from the medication or
Speaker 1: the fake medication, uh carry over effect, where the results
Speaker 1: of one study are are observed in a secondary study
Speaker 1: without realizing it. And then magnitude blindness, the tendency to
Speaker 1: become preoccupied with statistically significant results that never nevertheless have
Speaker 1: a small magnitude on effect. I feel like that comes
Speaker 1: into play a lot when I look at, um, some
Speaker 1: of this stuff that's new and that's being reported in
Speaker 1: the media. It's very exciting, right, you know, oh wow,
Speaker 1: look at this insight, and then when you get into
Speaker 1: the specifics of the study, it's just it's not that significant, right,
Speaker 1: doesn't quite match up to that snappy headline. All right,
Speaker 1: So how does science correct course? What can be done
Speaker 1: about these problems we've discussed? Well, um, just to talk
Speaker 1: briefly about the use of statistics and managing potential conflicts,
Speaker 1: those financial conflicts we we mentioned earlier, conflicts of interest. Um.
Speaker 1: The general idea that the expert put put forth is
Speaker 1: that we need to simplify, standardized and better enforced policies
Speaker 1: to manage financial conflicts of interest, and that science needs
Speaker 1: to keep a better eye on statistics, by which we mean,
Speaker 1: of course, the statistical validity and the statistical errors inherent
Speaker 1: in the system. Another thing is to encourage replication. And
Speaker 1: again this is from the Economist article quote. Some government
Speaker 1: funding agencies, including America's National Institutes of Health, which dish
Speaker 1: out thirty billion on research each year, are working out
Speaker 1: how to best encourage replication and growing numbers of scientists,
Speaker 1: especially young ones, understand statistics. Another area is allocating space
Speaker 1: and journals were uninteresting studies, which which is which is
Speaker 1: crazy because if you think about it in terms of, say, um,
Speaker 1: you know, a literary fiction publication, you would never in
Speaker 1: a million years have anyone suggests, hey, we should make
Speaker 1: room in this, uh this review for bad fiction. You
Speaker 1: know a certain amount that we're always just gonna include
Speaker 1: bad fiction. But the idea here is that scientific journals
Speaker 1: should allocate space for the less jazzy, the less sexy stuff,
Speaker 1: because that too is essential. Now I'm wishing for a
Speaker 1: journal called the humdrum Studies Journal, uninteresting Studies Journal. Now,
Speaker 1: another solution would be to tighten peer reviews, so perhaps
Speaker 1: dispensing with it altogether. And again that's from the Economist article.
Speaker 1: And so if you dispense with it altogether, what would
Speaker 1: do well? You would have post publication evaluation in the
Speaker 1: form of appended comments. And they say that that system
Speaker 1: has worked well in recent years in physics and mathematics.
Speaker 1: And lastly, policymakers should ensure the institutions using public money
Speaker 1: also aspect the rules. So picking up again to the
Speaker 1: potholes that we had mentioned, one of them is also
Speaker 1: skills neglect, and this is that human disposition to resist
Speaker 1: learning new scholarly methods that may be pertinent to a
Speaker 1: research problem. And so that would also factor into peer review.
Speaker 1: Is just making sure that while you're reviewing something else,
Speaker 1: but your own knowledge of the topic is up to snuff.
Speaker 1: And finally, when it comes to weird populations, I mean,
Speaker 1: the big thing is just to be aware of it
Speaker 1: too when you're when you're sampling, when you're using samples
Speaker 1: from the immediate collegiate environment to be aware of it
Speaker 1: and maybe be less cavalier about uh saying that you
Speaker 1: have identified something that is in you know, basic in
Speaker 1: general human nature. Of course, we should end this episode
Speaker 1: with the study of all studies, which is that there
Speaker 1: are too many studies. Yes, this was I believe that
Speaker 1: the time it was attention decay in science, um, which
Speaker 1: is snazzy. UM. And it basically just comes down to
Speaker 1: the fact that there are just so many studies coming
Speaker 1: out now in so many journals. They've just exploded since
Speaker 1: the earlier days um, in the twentieth century. Yeah, and
Speaker 1: it's hard for everyone to keep up with the studies,
Speaker 1: and also the older studies are getting lost in the
Speaker 1: fray of new studies. So um. Of course you know
Speaker 1: that building upon knowledge is really important in this discovery
Speaker 1: of truth. Right, And it's fair to point out that
Speaker 1: this paper should awesome also be analyzed, um because it's
Speaker 1: just one single study and the researchers mainly looked at
Speaker 1: very broad fields like chemistry and medicine. Indeed, trust but
Speaker 1: verify right, it all comes back. So so again, this
Speaker 1: episode wasn't It's not about you know, doubt everything, doubt
Speaker 1: every study, that comes out, doubt every the bit of
Speaker 1: science journalism that comes across your desk, But it's it's all.
Speaker 1: It's all information that's worth keeping in mind when you
Speaker 1: do engage with these studies. Uh, and something that you
Speaker 1: know that we like to keep in mind, you know,
Speaker 1: when we look at these studies in our research. Yeah,
Speaker 1: and we thought that this was pertinent information, especially when
Speaker 1: you consider how much data we are taking in every
Speaker 1: single day and all of the headlines that are connected
Speaker 1: to these studies and where they're coming from and how
Speaker 1: they're being pursed out. Indeed, Hey, in the meantime, if
Speaker 1: you want to check out more episodes of Stuff to
Speaker 1: Blow your Mind, most of which involve scientific studies of
Speaker 1: one type or another, you can head on over to
Speaker 1: stuff to Blow your Mind dot com, where you will
Speaker 1: find all those podcast episodes, all those videos, all those
Speaker 1: blog posts, you name it. And we know some of
Speaker 1: you are out there toiling away in the fields and
Speaker 1: the labs, scientific researchers. Do you have thoughts about this?
Speaker 1: If so, we would love to hear from you, and
Speaker 1: you can email us at blow the Mind at house
Speaker 1: to courts dot com for more on this and thousands
Speaker 1: of other topics, visit how stuff works dot com.