Medical and nutritional advice based on bad science is not a new phenomenon.
Archaic practices, such as “bleeding” a patient, were used for over 2,000 years. Metals like mercury and silver were prescribed for hundreds of years as a popular cure-all. Lobotomies were actually a fad in the early 20th Century – the inventor even won the Nobel Prize.
Hell, in the United States, heroin(!!) was sold over-the-counter as a cough suppressant until 1917. The list goes on.
But surely in modern times such deadly assumptions have been replaced with advice that’s been tested by the scientific method, right? Not exactly.
Unfortunately, bad science is still commonplace to this day – and it’s just not going away. Frankly, we’re sick of it, and to paraphrase the famous line from Network,
The most recent example came in the form of a recent report in which the World Health Organization (WHO) named processed meat a definite human carcinogen and red meat a probable human carcinogen.
We’ll get to the bottom of this report in a moment, but first, let’s take a look at some important examples of bad science in nutrition.
The Granddaddy of Modern Day Bad Science
We’ve shared this story before, but considering that it has almost single-handedly contributed to the out-of-control obesity rates in our country over the past 40 years, it’s definitely worth another look.
It all started with a “groundbreaking” study in the late 1950s by scientist Ancel Keys. His Seven Countries study found a correlation between saturated fat and heart disease in seven different countries. However, the study omitted all the data that didn’t support his theory, leaving only seven out of the original 22 countries included.
Keys’ findings were not universally accepted at first. Not by a long shot. One of the most vocal groups to come out and ask for more controlled studies was the American Heart Association (AHA). That was in 1957. By 1961, they had changed their tune.
There were no new studies conducted during that time. However, there were some changes on the board of the AHA. One of the additions: Ancel Keys.
From that point, the AHA started recommending a diet low in total fat (especially saturated fat and cholesterol) and high in carbohydrates from grains and polyunsaturated seed oils.
As a nation, we took that advice and ran with it.
Misinformation, Contradictions, and Poor Education
Ancel Keys is not the first, nor the last, example of bad science in nutrition. He is just the most popular. There will never be a shortage of misinformation and bias when it comes to what people both believe and teach. These days, everyone has something to say.
One of the more egregious examples of fear-mongering and misinformation in recent years was an article in the New York Times, reporting that meat-eating causes premature death and disease. The article was written in 2012, so is not associated with the more recent WHO report, but it is guilty of the same issues we are addressing here.
Gary Taubes, author of Why We Get Fat and Good Calories, Bad Calories, in which he points out the increasingly problematic trend of nutrition and health researchers using “pseudo-science” to support their hypotheses – no different than Ancel Keys.
We highly recommend you give it a read. Here’s a quote from the article:
“This is an issue about science itself and the quality of research done in nutrition. Those of you who have read Good Calories, Bad Calories (The Diet Delusion in the UK) know that in the epilogue I make a point to say that I never used the word scientist to describe the people doing nutrition and obesity research, except in very rare and specific cases. Simply put, I don’t believe these people do science as it needs to be done; it would not be recognized as science by scientists in any functioning discipline.”
These Government Organizations Are Still in the Dark Ages
The Department of Agriculture (USDA) and Department of Health and Human Services (HHS) recently issued the 2015 Dietary Guidelines for Americans, as they do every five years.
We reported in a previous post that when the 2015 Dietary Guidelines Advisory Committee submitted their draft report to the USDA and HHS, the biggest revelation was “Cholesterol is no longer listed as a nutrient of concern.”
However, this was not a revelation to experts in the nutrition field. We ditched that “egg whites are better for you” nonsense a long time ago, but why hasn’t the general public? Because they’re listening to the nutrition advice given to them by their government.
As the Washington Post reported when they first broke the story of the Advisory Committee’s report, currently, the National Institutes of Health spends about $1.5 billion annually on nutrition research, an amount that represents about 5 percent of its total budget.[Tweet “Nutrition is five percent of the National Institutes of Health budget.”]
Let that sink in.
1) As you know, we say nutrition is 75% of one’s health, yet the government only invests 5% of its funds toward it. That’s very skewed.
2) This low investment is one reason there isn’t better science. More money means more funded studies. More studies, ideally, means more data points that move us closer to truth.
The US government has continued to be slow on the uptake when it comes to the latest research on nutrition. The reprieve of cholesterol is a win for truth and wellness, but they continue to treat all fats as equal in the new guidelines, so they are still well behind the times.
Since the first Dietary Guidelines report was issued in 1980, Americans have been urged to avoid fat and animal products as the best way to avoid heart disease and obesity. We listened, yet obesity and heart disease rates have exploded.
The new guidelines have changed their stance on total fat consumption, which is certainly a step in the right direction, but continue to ignore the fact that there is no evidence against saturated fat.
The change in perspective on cholesterol is definitely promising. While it’s easy to piss on the government (and often deserved), it’s not necessarily easy for them to make good nutrition policy either.
Bad science and confusing information is their source also. So we’ll give them a bit of a break here – so long as you promise to ignore their guidelines!
So What Does The WHO Report Say?
Okay, okay, you’ve been patient. And frankly, you’ve already seen the sensationalized headlines. But what does it all mean? And what’s our beef with it? First things first, you can view the published report here.
- The World Health Organization estimated that 50 grams daily of processed meat or 100 grams daily of red meat might increase the risk of colorectal cancer by 18 percent and 17 percent, in that exact order.
- Processed and cured meat has been classified as a ‘definite’ cause of cancer (or Group 1 carcinogen) – the same group that includes asbestos, alcohol, arsenic, and tobacco.
- Red meat is a ‘probable’ cause of cancer (or a Group 2a carcinogen) – the same group as long-term night shifts.
The Scientific Method Revealed
Tom Naughton’s “Science for Smart People” gives an incredibly accurate (and hilarious) assessment of the quality of research being done in the nutrition industry. You can check out the video below:
I’m going to break down some of the major points contained in the video and use those points to see if the WHO report stands up to the criteria.
- Make observations
- Form a (testable) hypothesis
- Conduct experiments
- Collect data from experiments
- Reach a conclusion based on the data
*Note: The hypothesis is not valid, unless the results are consistent and repeatable.
Naughton recommends you ask several important critical thinking questions when reading the findings of these studies. Here are some of the ones that best apply to the WHO Report:
Is this an observation or clinical study?
– Observational studies
Clinical trials are experiments. The best type of clinical trial is the Randomized Controlled Trial (RCT) in which people are randomly assigned to one of the trial groups. Each group is then given just one thing different. Everything else is the same.
Group 1 = thing A
Group 2 = thing B
Group 3 = no things (they are the control group)
The “thing” might be a specific diet, a supplement, or the number of alcoholic beverages to drink each day.
Now we can collect data from each group. At the end of the trial, we can review the data to see if our hypothesis was correct.
Clinical trials are useful for steps 3, 4, 5, and 6 of science. They help prove or disprove hypotheses.
Observational studies are much different. There is just one group of people and they are observed for a period of time. Data is collected about all those people, usually by sending them periodic questionnaires to answer. Researchers can then review that data and look for patterns.
Here’s a very important thing to know: Observational studies can never prove that A causes B. They can only say A seems to be connected with B.
Said another way, observational studies cannot draw definitive conclusions. They are only useful for steps 1 and 2 of science. They can help us form hypotheses.
Verdict: The WHO Report is an observational study.
What was the difference?
There are two different ways to present data in a report, and you need to be able to distinguish at which one you’re looking. They paint a very different picture.
- Absolute change = subtraction (new quantity – original quantity)
If you had one penny, and someone gave you a second penny, you now have two pennies. Two pennies (new quantity) minus one penny (original quantity) = one penny. The absolute change is one cent.
- Relative change = division (absolute change / original quantity) multiplied by 100 to make it a percentage.
In the same example, one penny (absolute change) divided by one penny (original quantity) multiplied by 100 = a relative change of 100%.
See the difference? If you had one penny, and someone gave you a second penny, sure, you have a 100% increase in money, but the reality is that you now have two cents instead of one cent – not that big a difference after all.
So how does this relate to the WHO report?
The statistics don’t mean that eating 50 grams of processed meat daily will cause you to have an 18% total chance of developing cancer — it means you’re 18% more likely to develop cancer, relative to whatever your initial, absolute risk already was.
For example, a person with an average risk of colorectal cancer has about a 5% chance of developing colorectal cancer overall. If you ate 50 grams of processed meat every day, your risk would increase to 6%, or an absolute risk of 1%. But that number isn’t going to sell many newspapers or get a lot of clicks, is it?
Verdict: The WHO Report uses a sensationalized relative change over the much more conservative absolute change
Did researchers control the variables?
The problem is confounding variables, or variables that the researcher failed to control.
Health consciousness is a confounding variable!
A study shows that people who eat red meat are more likely to die sooner.
People who eat meat (A) —> are more likely to die sooner (B)
That study of course leads to ridiculous headlines like this: “Red meat increases death, cancer and heart risk” (BBC).
The people who ate red meat were also the people who smoked, didn’t exercise, drank more alcohol, visited the doctor less frequently, and were generally unhealthier people. Those who ate less red meat were more like your friend who always buys fresh juice from Whole Foods after yoga.
It’s far more likely that “being a generally unhealthy person” (C) leads to both “eats more meat” (A) and “more likely to die sooner” (B).
Also worth mentioning, in the WHO study, grass-fed, organic, locally-sourced red meat was lumped into the same group as cheap hot dog meat from a fast food joint.
“There’s also the fact that red meat suffers from an “unhealthy user bias.” Most heavy red meat eaters aren’t sprinting, lifting weights, and going for walks every day. They’re eating their meat between buns and with fries. They’re getting their red meat from Burger King or the 7-11. They can try to control for most of these associations, but it’s impossible to account for everything.”
-Mark Sisson, Mark’s Daily Apple
Just because two traits are correlated does not mean one is causing the other. That is known as association.
If your favorite team wins every time you wear your lucky underwear, these two things are correlated, but does that mean that your underwear is literally the cause of your team’s winning streak? Doubtful. There are probably some other variables at work there.
Check out this hilarious website that either proves that Nicolas Cage has single-handedly caused hundreds of drowning deaths in the U.S., or… that association is NOT causation.
One of our favorite experts has something to say about the degree of association found in this report:
“Even allowing for the weakness of observational studies, and the unreliability of dietary questionnaires, and the notion that food consumption can be a marker not a maker of health, and the whole dietary intake that has not been taken into account and the ignorance of the chasm between real and processed food, this is still association, not causation. I always wish that these huge and expensive studies would ask what colour socks the participant is wearing. I bet I could find an association between red sock wearing and one type of cancer if I looked hard enough. Would the headline be red socks cause cancer?”
Verdict: The WHO Report ignores confounding variables and relies on association, not causation.
Be VERY Skeptical of the Conclusions Drawn from these Studies
Naughton references a great article in The Atlantic, titled Lies, Damned Lies, and Medical Science. In the article, Dr. John Ioannidis claims:
- Almost 90% of published medical information is flawed
- 80% of the conclusions drawn from observational studies are wrong
- 20-25% of the conclusions drawn from clinical studies are wrong!
How can that be?
Researchers were frequently manipulating data analyses, chasing career-advancing findings, rather than good science. Sometimes, they’re just in the pocket of powerful organizations like Monsanto. Read our article on it here.
There’s no evidence that this is the case with the WHO study, but these are some pretty alarming statistics, even at a fraction of the amount that Dr. Ioannidis suggests. And the World Health Organization is no stranger to blurring the line of impartiality…
WHO is increasingly relying on ‘partnerships’ with the industry, instead of maintaining neutrality, like it always has done, to fill holes in its budget.
Does Eating Meat Cause Cancer?
Really, we still don’t know. That’s the challenge with bad science. It sets us down one path, when we’re really still standing at the intersection unsure of what to do.
Let’s eat all things in moderation (except veggies! Eat as many of those as you like), take care of our bodies – both physically and mentally – and keep our eyes and ears open for actual clinical studies.
“At this point, given what the research indicates, I do not feel that modest consumption of cured or processed meat is likely to pose a significant health risk, provided you are doing other things right (i.e., nurturing your gut microbiome, eating nutrient-dense, real foods, exercising, etc.). I think there is even less evidence suggesting that we should limit consumption of fresh red meat, especially when it is cooked using gentle methods (rather than charring it) and when you eat ‘from nose to tail.’”
So Who Can You Trust?
More and more research is being published every day. The key is to avoid falling into the trap of blindly following other people’s thinking without bias.
Journalism will never be a cautious profession as long as its aim is to find and communicate events that are of interest to broad sectors of society. –NCBI
You should remain open-minded and afford yourself the option to change your mind at any time.
We realize the irony of this section – we are offering advice while telling you to be careful who to trust. We do our best, here at Factor, to provide the most accurate and science-backed nutritional information we can find. We have a nutrition and performance advisory board with some brilliant people in the field of fitness and nutrition who help keep us in check on everything.
We also sort through the noise out there to identify a group of people who have stood the test of time in handing out fantastic evidence-based advice. These are people like Chris Kresser, Robb Wolf, Gary Taubes, Mark Sisson, and Zoë Harcombe – to name just a few.
That being said, you should absolutely do your own research on the matter. If you agree with what we have to say, please give us a share, so we can spread the word.
Either way, let us know your thoughts in the comments below. We encourage healthy discussion!
Thanks for bearing with us while we got that off our chest. This is a topic you have been asking a lot about, and we’ll continue to keep you in the loop on any new information as “good” science continues to clear up the picture for us all.
We know that at the end of the day, you come to us for actionable nutritional advice that will help you feel fitter, healthier, and perform at your best. The good news is that it’s not complicated!
It’s actually pretty simple.
You just need to Eat Real Foods and avoid the worst foods. Start there and you’ll notice instant results – regardless of your fitness goals. But what does that mean exactly? Here’s a quick and easy guide to eating cleaner and having limitless energy.