“Back in 1995, Claude Steele published a study that showed that negative stereotypes could have a detrimental effect on students’ academic performance. But the big surprise was that he could make that effect disappear with just a few simple changes in language. We were completely enamored with this research when we first heard about it, but in the current roil of replications and self-examination in the field of social psychology, we have to wonder whether we can still cling to the hopes of our earlier selves, or if we might have to grow up just a little bit.”
This podcast examines the nature of scientific knowledge and how it is produced. What is “good science” and how is it undermined by incentives and the process itself?
“Academic research stands accused of turning a blind eye to dodgy data, failing to reconcile contradictory findings and valuing money over knowledge. We examine the criticisms, which go the very heart of our pursuit of knowledge.”
Click on the image for the full cartoon.
“An influential psychological theory, borne out in hundreds of experiments, may have just been debunked. How can so many scientists have been so wrong?”
“And yet, it now appears that ego depletion could be completely bogus, that its foundation might be made of rotted-out materials. That means an entire field of study—and significant portions of certain scientists’ careers—could be resting on a false premise. If something this well-established could fall apart, then what’s next? That’s not just worrying. It’s terrifying.”
“Nosek recently lead a project in which 270 scientists sought to replicate 100 different studies in psychology, all published in 2008 — 97 of which claimed to have found significant results — and in the end, two-thirds failed to replicate. Clearly, some sort of course correction is in order.”
This is a great podcast that gets into some of the issues and challenges with constructing knowledge in the human sciences. Though we differentiate between the human and natural sciences, both use similar processes to construct experiments and rely on similar reasoning processes to construct knowledge. One key idea in both types of science is the idea that experiments can be reproduced and results can be replicated in order to validate a study’s findings. What happens when you can’t reproduce a prior finding? Were the original results fraudulent? Poorly constructed experiments? Was the new experiment faulty? The answer to these questions is complicated but this podcast delves into some of those issues.
“Lots of psychology studies fail to produce the same results when they are repeated. How do scientists know what’s true?”
Reproducibility is a central part of producing knowledge in both the natural and human sciences. Production of knowledge in the sciences is also supposed to be “objective.” What happens when you can produced a statistically significance conclusion but it turns out to be false? What happens when other scientists cannot confirm your conclusions when they try to recreate your experiments? Is it in any way practical to redo others’ experiments consistently?
“How much of published scientific research is false? Scientists are trying to figure it out.”