Why Most Health News Is Trash

Whenever you read any health news publication you’re greeted with headlines that share several familiarities. They usually look like this:

“New study suggests {insert ingredient here} may be linked to {insert condition here}”

or something like…..

“{insert benign activity here} may lead to {insert unusual condition here}”

The hook is almost always an implicit suggestion that some food or activity has a strong correlation with some other outcome, and if you only did more of that activity or consumed more of that food, then you’ll see a corresponding decline like the results presented in the article.

1+1=2, right?

Well, no. Sorry. While those studies do convey some truth about a link between a particular nutrient and outcome, the implied meaning about some new direction you ought to take with your health is all but nil.

News headlines about health appeal to our instinct to categorize information, but do not correlate with scientific reality. Here are the reasons why.

Reason 1: Insufficient Sample Size

Lots of observed medical effects completely disappear when the same study is performed on a large group of people, even if the effect is statistically significant on a small number of people.

Let me give you an example.

A study published in a 1992 journal of Science found that a variation of the ACE enzyme significantly reduced someone’s chance of heart disease. The study was conducted with 500 participants, and the results were statistically significant.

Hooray! A breakthrough in medical progress, right?

Nope.

When the experiment was conducted on 10,000 people instead of 500 the effect disappeared. And this phenomenon is not unique. In many genetics studies significant results in small sizes turn out to be insignificant when studied on larger numbers of people.

Reason 2: New Research Isn’t All That Useful

The news world loves “new” studies that appear to turn over some undiscovered leaf because they can capture attention if they have unusual results. The problem is that for most health problems, it takes years of research and hundreds if not thousands of studies performed on a particular topic before a consensus is reached. And this says nothing of the quality of research performed.

Research on a new scientific topic is a lot like the 1.0 release of a new piece of technology. It might show a little bit of promise, but it’s also buggy and lacking refinement. The same is true in science. It often takes years of research to lay the groundwork for a ground breaking study to come along that definitively answers important questions. Dozens or hundreds of studies are usually needed to pick off nagging issues that contaminate the validity of research results. And by the time these studies come in, the overall perspective on the condition is usually mottled enough that news headlines devoted to the subject are likely to be exaggerated.

Reason 3: Research Results Usually Don’t Translate Into Practical Action

Even if the latter two points have been satisfied, health research results don’t always imply you should take a particular course of action. A good example would be studies performed on cancer prevention. A lot of them involve taking an isolated line of cancer cells and then gently treating them with a concentrated extract of some chemical or nutrient and seeing  what happens.  If scientists pump enough of the extract into the cells and replication begins to recede, the news world rejoices with a new discovery that “X ingredient cures Y type of cancer.”

If you think that’s important for what you need to change in your diet, let me ask you something: when’s the last time you drank a glass of de-glycerized licorice root concentrate?

There are stark differences between a controlled laboratory setting with isolated variables and the nutritional effect of eating more of a store-bought food and ingesting it into your body like normal humans do. The benefit you’ll experience is going to be much more modest, if there’s anything at all.

Many articles written about these studies don’t convey the sheer amount of the chemical involved needed to observe the change. For example, a lot of studies have demonstrated the long-term health benefits of drinking green tea. Those are real. But in most of those studies the amount of green tea extract required to create the measured result is far in excess of what would be considered normal consumption.  Many times you’d have to drink 4-6 cups per day for several years to equal the nutritional density of the green tea supplements used in the study.  Not even tea-guzzling Okinawans drink that much.

Conclusion

This rant isn’t a vote against healthy living, or the benefits about understanding the core principles of good health and nutrition. It’s just the opposite, since the overall effect of health news consumption is to cultivate a “well-informed ignorance” about what scientific research does, and does not tell you about the benefits of healthy food.

To really understand the scope of the health news you read, you have to realize that the observed effects in a study are almost always going to be smaller when they’re incorporated into your daily routine because life is much messier than a laboratory experiment.

References:

1. Gelman, Andrew, Weakliem, David. “Of Beauty, Sex, and Power.” The Scientific Research Society. July 2009, pgs. 308-316. URL: http://www.stat.columbia.edu/~gelman/research/published/power4r.pdf
2. Pickrell, Joe, Barrett, Jeff, MacArthur, Daniel, and Jostins, Luke. “Size Matters, and Other Lessons from Medical Genetics.” Genomes Unzipped. Nov, 23, 2011.
3. URL: http://www.genomesunzipped.org/2011/11/size-matters-and-other-lessons-from-medical-genetics.php?utm_source=feedburner&utm_medium=feed&utm_campaighttp://www.genomesunzipped.org/2011/11/size-matters-and-other-lessons-from-medical-genetics.php
4. Cambien, Francois, et. al. “Deletion Polymorphism in the Gene for Angio-tensin Converting Enzyme is a Potent Risk Factor for Myocardial Infarction.”  Letters to Nature. October 15, 1992, pgs 641-644.
5. Bechtel, Jonathan. “Green Tea: Health Benefits, Catechins, and Cancer Prevention.” Health Kismet. January 19, 2012. URL: http://blog.healthkismet.com/green-tea-health-benefits-catechins-cancer-prevention