How one flawed study and irresponsible reporting launched a wave of CTE hysteria

Interesting if not controversial piece about the science behind concussion research and professional football. This raises interesting questions about the extent to which “good science” is even possible in a situation like this when brains can only be examined posthumously. There is definitely a selection bias here because people only want to have their brains examined if they believe they suffer from the condition.

When we dug into the methodology, we were floored. The study was so badly flawed that it was nearly worthless. But that’s not what had been reported in practically every major media outlet in the world. Thanks to the barrage of sensationalist coverage, the “110 out of 111 brains” story had turned into a wildfire, and we were standing around with a couple of garden hoses, telling everybody to calm down.

How can we use math to help us understand sports?

There are some interesting debates raging within the sports community about the ways in which statistics can help us understand athletic performance and the value of different players to a team. These statistics also are used to evaluate what are effective strategies and which are not.

Though some of the debate is about whether we should or should not rely on these statistical models, there are some interesting differences among those models themselves. Each model relies on different assumptions and maps the reality of the game differently. Sometimes, as in the case discussed in the article below, different models give us wildly different answers about a player’s value. Which is correct? What does this case tell us about the ability and limits of using math to understand reality? Is it possible to resolve this debate?

How does the quote below apply to this case?

“A man with a watch knows what time it is. A man with two watches is never sure.”

Implicit bias and the NFL draft Teams don’t recognize how unconscious attitudes about race affect which players they select

“Even in an industry where minority workers sometimes appear to be favored for highly desirable jobs,” the two concluded, “employers may still fall prey to symbolic discrimination, relying on deeply embedded stereotypes about minority groups during the interview process.”

In N.F.L., Deeply Flawed Concussion Research and Ties to Big Tobacco

This article connects to some interesting TOK issues. Clearly we can discuss the ethics, or lack of ethics, in the NFL’s manipulation of data to disprove conclusions that undermine its business.

This also illustrates how math can help us understand and possibly prove complex issues like the connection between football and health issues like concussions and CTE.  Rather than observing or intuiting a causal relationship between two phenomenon, we have to use math along with the methods of proof in the natural sciences to establish truth and construct knowledge. By misrepresenting data, one might reach incorrect conclusions, which seems to have been the case here.

A second article about how flawed data undermines our ability to construct knowledge.

“Researchers primed to believe that the NFL has concussions under control, a data set that’s missing important information, and publication in a journal edited by a consultant to the NFL — it looks more like an attempt to create evidence for a predetermined message than good science. But even if we throw out these studies, we can’t yet conclude that football inevitably leads to lasting brain damage.”

The Ethics of Watching Football

1. Room for Debate: Is It Wrong to Watch Football?

This first link is from the New York Times’ Room for Debate series. Here, four experts discuss the question about whether it is ethical to watch football.

“How can fans enjoy watching a game that helps ruin players’ lives?”

2. The Ethicist: Is it Wrong to Watch Football

From the New York Times’ series, The Ethicist. 

“What you are concerned about involves one disquieting aspect of one specific sport. You want to know if it’s ethically acceptable to watch a game that is dangerous to the athletes who participate. And the answer to that query is yes.”

3. Aaron Hernandez suffered from most severe CTE ever found in a person his age

From the Washington Post, an article that details that brain damage suffered by a young and prominent NFL star who was also convicted of murder. If this type of brain damage is possible from playing the sport, should it be legal? If it legal, with the consent and full information of those playing, is it ethical to watch this sport?

4. The Federalist: Now That We Know Football Hurts Athletes, Should We Keep Watching?

“It’s time for football fans to consider the morality of a sport that turns young athletes into middle-aged corpses, racked by dementia and disabilities.”

“How long can an activity that may carry with it the likelihood of an awful life-shortening ailment continue to hold the imagination of the country? Those who believe football is too big and too popular to ever be cast aside should remember that only 80 years ago, boxing reigned alongside baseball as the country’s only true national sport. Even a half century ago, when Muhammad Ali was heavyweight champion, boxing was still immensely popular even if, unlike in previous generations, the percentage of youngsters who boxed was tiny. Today, it still exists and manages to hold a niche of the sports market, but it is a marginal endeavor derided for its brutality that increasingly few American care about.”

5. Reason Magazine: Is Watching Football Unethical?

From a Libertarian perspective, how should we view the ethics of watching football?

6. I’m the Wife of a Former N.F.L. Player. Football Destroyed His Mind.
He chose the sport, but he did not choose brain damage.


N.F.L. Announcers Are Bad at Math, Too

What does this article tell us about people’s motivation to take “correct” actions? What happens when math says one thing but our emotions tell us another? What if the agreed upon consensus correct answer is in fact wrong?

“It’s not that coaches don’t know the math — rather, it seems they don’t want to be criticized. If a coach does the expected and sends out the punt unit on fourth down, and then his team goes on to lose, players are blamed for the defeat. If the coach orders a conversion attempt that fails, the coach is blamed for subsequent defeat.”®ion=top-stories-below&WT.nav=top-stories-below&_r=0

4th Down Bot. Live analysis of every N.F.L. fourth-down decision

4th Down Bot copyThis is a clever program that does an analysis of every 4th down play in every professional football game. It determines based on mathematical expected value whether teams should go for it, punt, or kick a field goal. It breaks down the math behind its decision making. What’s interesting is how often the mathematical decisions are not the ones followed by the people on the field. Who is right in a case like this? What happens when the “common sense” approach is different from the mathematically “true” approach?

How Oregon Coach Chip Kelly Can Spark ‘Moneyball’ Revolution In NFL

“Those fourth down calls epitomize Kelly’s aggressiveness but what the average football fan doesn’t realize is that Chip’s play-calls (the fourth down tries, fake punts, two-point conversions, etc.) are almost always the correct mathematical decision. Like Paul DePodesta and Billy Beane did in baseball, Kelly’s genius comes from exploiting arithmetic that other coaches are too naïve to acknowledge.”

Using Math to analyze baseball. Sabermetrics

Popularized in the book and later on the movie, Moneyball, Sabermetrics is “the search for objective knowledge about baseball.’ Thus, sabermetrics attempts to answer objective questions about baseball, such as ‘which player on the Red Sox contributed the most to the team’s offense?’ or ‘How many home runs will Ken Griffey hit next year?’ It cannot deal with the subjective judgments which are also important to the game, such as ‘Who is your favorite player?’ or ‘That was a great game.'” -Bill James.

Sabermetrics has caused tremendous controversy among sports analysts and enthusiasts because of the positions they take on questions such as: How do we reconcile mathematical knowledge that contradicts our intuition? What if our eyes tell us one truth and our numbers tell us a different one?

Some people complain that the reliance on numbers takes away from the “magic and mystery of the game.”

The baseball organizations themselves have adopted the data driven approach to analyzing players. With the success of the Oakland Athletics (a team that was an early adopter of the mathematical methods known as Sabermetrics) and later on the Boston Red Sox, most if not all teams use these analytical methods.

The first link below is an introduction to the basics of sabermetrics and below that are two disagreeing with its use.