What Afghanistan shows is that we need a new definition of expertise, one that relies more on proven track records and healthy cognitive habits, and less on credentials and the narrow forms of knowledge that are too often rewarded. In an era of populism and declining trust in institutions, such a project is necessary to put expertise on a stronger footing.
Tetlock and the Taliban
How a humiliating military loss proves that so much of our so-called “expertise” is fake, and the case against specialization and intellectual diversity
The American-led coalition had countless experts with backgrounds pertaining to every part of the mission on their side: people who had done their dissertations on topics like state building, terrorism, military-civilian relations, and gender in the military…Meanwhile, the Taliban did not have a Western PhD among them.
This podcast explores the potential cognitive and developmental issues that air pollution can have. The best part of this episode though gets into the very clever methods the researchers employ to find natural “experiments”.
Before listening to selections from this episode it might be interesting to ask students to design a hypothetical experiment to evaluate whether the hypothesis explored in this episode is true and then listen to some of the actual methods employed to then examine why those choices were made.
Of particular interest was a research study discussed around the 26 minute mark in which a government policy in China provided an excellent “natural experiment” to help think about the impact of air pollution. Part of evaluating this topic could also be a discuss of ethics in experimentation when kids ultimately decide that a good experiment would be to pollute the environment of one group and not another to see what happens.
Air pollution is estimated to cause 7 million deaths a year and cost the global economy nearly $3 trillion. But is the true cost even higher? Stephen Dubner explores the links between pollution and cognitive function, and enlists two fellow Freakonomics Radio Network hosts in a homegrown experiment.
Attached are some passages from the book, Cribsheet by Emily Oster, an economist who wrote a data-driven guide to parenting. I put together some interesting passages from the introduction and from one of the chapters that does a nice job contextualizing the concepts of data driven decision making, what a good study is, the limits of those studies, and the ultimate uncertainty of all the knowledge produced using data.
Meaningful connections to constructing knowledge and data collection in the human sciences (particularly economics), natural sciences, and cognitive biases. Also deals well with problems of sorting out the differences between correlation and causation.
Generally great book for parenting, not just for its TOK connections.
How can you identify a good study? This is a hard question. Some things you can see directly. Certain approaches are better than others – randomized trials, for example, are usually more compelling than other designs. Large studies tend, on average, to be better. More studies confirming the same thing tends to increase confidence, although not always – sometimes they all have the same biases in their results
In November, 2020, Science Magazine published a study (link to the actual paper) evaluating the impact on younger female scientists of having female or male mentors.
Here are some sections of the paper:
“Our gender-related findings suggest that current diversity policies promoting female-female mentorships, as well-intended as they may be, could hinder the careers of women who remain in academia in unexpected ways…Female scientists, in fact, may benefit from opposite-gender mentorships in terms of their publication potential and impact throughout their post-mentorship careers.”
The paper drew rather swift criticism and outrage and then outrage at the outrage. Out of all the…outrage some interesting issues arise:
Was the methodology of the study sound? Also worth noting that this was a statistical study that found a correlation, how did they conclude a causal link? Related question is whether people are criticizing the methodology because they are interested in rigorous science or because they want to undermine a study whose conclusions they don’t want to believe.
More importantly though, even if the study were done soundly, is this an appropriate question to study? Does the question/issue being studied violate some ethical standards? If a study turns out to be true but causes harm, should it be published or even explored in the first place? Are some questions off limits?
If only one paper finds an association or correlation can we call this scientific knowledge?
How do we, as individuals or communities of knowers, respond when a scientific study violates our personal or political beliefs?
This last question above is basis of the backlash against many of the study’s critics.
Here’s one article that goes over the issue:
A Study Claimed Male Mentors Are More Helpful to Women Scientists—and It Did Not Go Over Well
Great episode with some solid TOK connections. At around 30:55, the host talks to economist Ed Glaeser about the value and importance of cities. The arguments he present connect well to concepts of personal and shared knowledge. Cities allow for innovation and progress because of the ease of sharing knowledge.
DUBNER: You call the city “our greatest invention,” which means you’re putting it ahead of mathematics, the computer, antibiotics, rum-raisin ice cream. How so? How is the city the greatest human invention? GLAESER: It is the machine that makes all the other inventions possible, right? And the reason for that is that almost every one of the inventions that you raise, whether it’s mathematics and its development, whether in classical Greece 2,500 years ago or in the House of Wisdom in Abbasid, Baghdad, 1,200 years ago, it is human connections that make that creativity possible. Almost nothing that we have done as a species is a solo creation. We collaborate. We learn from each other. We steal each other’s ideas with some degree of regularity.
Further on, around 39 minutes, the host talks to economist Jennifer Doleac about crime statistics. What I love here is the way in which she talks about data and possible explanations for observed phenomena. All of this is a great representation of the methods of the human sciences and of economics in particular.
Homicide, it’s a bit of a mystery. And I think the mystery is compounded by the fact that we’re seeing different effects in different cities. So, it’s hard to come up with a really clear story. It’s possible that some of this is gang violence. And so, if people aren’t out and about, then there aren’t witnesses and bystanders who might deter violent crime or violent confrontations among people that perhaps don’t care about stay-at-home orders. Something else that has been written about quite a bit now is what happens to domestic-violence rates. We see domestic violence rates increase quite a bit. Some of those surely are turning into homicides. But that probably is not explaining all of the potential effect on homicides in these cities.
From pages 154-155 of Frank Knight’s 1940 review essay titled “‘What Is Truth’ in Economics?” as this essay is reprinted in Knight’s 1956 collection, On the History and Method of Economics:
Economics and other social sciences deal with knowledge and truth of a different category from that of the natural sciences, truth which is related to sense observation – and ultimately even to logic – in a very different way from that arrived at by the methodology of natural science. But it is still knowledge about reality.
The fact that economists, with relatively rare exception, cannot conduct controlled laboratory experiments which allow a focus on the behavior of a small handful of variables does not render the knowledge arrived at by economic scholarship – observation, research, and reasoning – unscientific…
To fancy that one can gain an adequate understanding of the workings of the economy merely by carefully observing and measuring the relatively few objective pieces of quantified data that are typically available to economic researchers – “the” unemployment rate, “the” four-digit concentration ratio of this and that industry, “the” Gini coefficient for this and that country…is foolish. All such observable empirical facts are the results of vast and complex plan formation and modification and human interactions. Observable facts about the economy have no meaning on their own, and they are not – or ought not to be – the subject-matter of economics.
Psychiatric therapies have never been atheoretical – psychiatrists have always justified their treatments with some school of thought: Freudian psychodynamic theories placed blaPsychiatric therapies have never been atheoretical – psychiatrists have always justified their treatments with some school of thought: Freudian psychodynamic theories placed blame on early childhood and subconscious urges; behaviourism justified the application of pain to try and train people out of wrongthink; and more recently, chemical imbalance theories were used to advertise pharmaceuticals, despite the narrative of simple dopamine and serotonin dysfunctions having been long dismissed in academic circles…Psychiatry today can be considered a discipline in crisis, surviving only because psychological and pharmaceutical treatments are effective for some people, some of the time, and so we still need them…
In 1973, after gay rights protestors had stormed an APA meeting…the APA called a vote to reconsider homosexuality’s status as a mental disorder. By a 58% majority, it was struck off.
Voting to decide whether a condition is a disease is not the sign of a mature science, yet to this day consensus alone is the deciding factor of whether a mind is deemed disorderly…
Nevertheless, this reference to evolutionary function is the only way to define function by an objective biological process rather than relying solely on subjective norms and cultural values – or indeed a vote.
F.A. Hayek on the importance of humility and limitations of Economics and the social sciences in general.
I feared…the Nobel Prize confers on an individual an authority which in economics no man ought to possess.
This does not matter in the natural sciences. Here the influence exercised by an individual is chiefly an influence on his fellow experts; and they will soon cut him down to size if he exceeds his competence…
I am not sure that it is desirable to strengthen the influence of a few individual economists by such a ceremonial and eye-catching recognition of achievements, perhaps of the distant past.
Interesting post about the nature of language and how it can misconstrue a complex reality and ultimately lead to misunderstandings and poor policy actions by governments.
Muddled thinkers confuse the world of our senses with the way in which it is depicted in language.
Yet as is true of all beneficial institutions, language is imperfect – it has, some might say, its ‘costs.’ Among the ‘costs’ of language is its tendency to cause us to suppose that the abstractions that we describe with words possess a concrete reality that these abstractions don’t possess. 🔥🔥🔥 [flames are my addition]
There are many questions to which we do not have scientific answers, not because they are deep, impenetrable mysteries, but simply because they are not scientific questions. These include questions about love, art, history, culture, music-all questions, in fact, that relate to the attempt to understand ourselves better. There is a widespread feeling today that the great scandal of our times is that we lack a scientific theory of consciousness. And so there is a great interdisciplinary effort, involving physicists, computer scientists, cognitive psychologists and philosophers, to come up with tenable scientific answers to the questions: what is consciousness? What is the self?