Epidemiology refers to a type of science that is essentially observational and examines group health. It looks at a group of people, some aspect or aspect of their diet or lifestyle and some outcome such as disease. Then it tries to determine what variable (or variables) of diet or lifestyle correlate with some outcome. And epidemiology is crap science so far as I’m concerned. Now let me tell you why.
Table of Contents
- The First Epidemiological Study
- The Egg Study
- Memory Based Food Recall in Epidemiology
- Memory Based Food Recall is Terrible
- Memory Based Food Recall is Terrible Part 2
- Everything is Multi-Factorial
- Overall Lifestyle Patterns
- Bias in Epidemiology
- Epidemiology is Crap
The First Epidemiological Study
In college I took a history of medicine class and learned about what I think the first epidemiological study was. This was hundreds of years ago where it was observed that people in one part of a city (in France or England maybe?) got sick and those in another didn’t.
So people wanted to find out why. As it turned out, each part of the city got all of it’s water from a different well and this was the big variable between the two areas. So it was logical to look at their water supply for initial clues.
And it turned out that one part of the city was mixing sewage with the water supply. Shockingly (or not), those people were getting sick. Voila, epidemiology is born along with this impressive practical conclusion:
Don’t shit in your water supply.
But that’s kind of where it started. That brings us to today. Where every new epidemiological study seems to contradict the last. And where single nutrients or aspects of lifestyle get blamed for some disease. And where nutritional policy is set based on these observations.
Observations from a science that is not just weak. It’s crap.
The Egg Study
The epidemiological study that kicked this off was a long-term study of 30,000 people over a span of 13 to 30 years. That’s a lot of people for a long time which should make it valid, right?
It concluded that a little under 2 eggs per day raised the risk of heart disease by 17%. And that each additional half egg raised it more.
Case closed, right? Eggs are bad again. Or are they?
Because for some reason I came across an article published on Undark.org that addressed this. And it spurred this piece. Since I have zero clue how valid or not the site is, I’ll be focusing on the researchers and studies they referenced rather than the Undark piece.
You will notice similarities in their and my writing simply because I’m quoting the same primary references as they did. But I will be adding much of my own thoughts to that.
Memory Based Food Recall in Epidemiology
Most epidemiological studies used what is memory based food recall as part of it’s “method”. This means that subjects are asked to fill out a food diary based on memory. What did you eat today? A week ago? A month ago? Last year? 10 years ago?
Can you see the problem here?
Outside of athletes who eat the same thing every day, who the hell can remember what they ate a week ago, much less a month or decade ago. Even if they could remember, would that brief memory represent the past 10 years of food intake? Of course it wouldn’t.
And yet that is what this “research” is usually based on: asking people what they ate over some time frame or some time ago and hoping it’s accurate.
So many studies will ask the subjects what they ate at the beginning, part way through, at the end or some combination. Many don’t do that.
In the case of the egg study, the subjects did a single food diary at the start of the study and never again. The researchers seem to have assumed their diet didn’t change for the next ONE to THREE DECADES.
They couldn’t even bother to ask again at the end for any sort of food recall. One food recall diary at the beginning of the study. That’s it. Do you eat now what you ate 30 years ago? I doubt it. It’s pitiful.
But it gets even worse than that.
Memory Based Food Recall is Terrible
The simple fact is that memory based food recall is a terrible method in and of itself. We’ve known for decades that people are terrible at reporting their food intake. Most underreport ranging from 20-50% of total calories.
This is why people will report eating only 800 calories and not losing weight. They are actually eating double that. Lean people do it, obese people do it, active people do it and inactive people do it. Even dietiticans suck at it. Everybody is terrible at doing it from memory.
In addressing this, the researchers Archer and Pavla have shown that self-reported intakes are physiologically implausible. On average, they are nearly 50% below the calories needed for survival. The data can’t possibly be correct because the people would all be dead if they truly ate that little every day.
But this has another implication.
If self-reported calories are 50% lower than reality, the reported intakes of protein, carbohydrate, fat or any given micronutrient will be as well. It will all be wrong. You can’t even assume that they are all 50% low because people mis-report some nutrients more than others. The whole data set is garbage because everybody sucks at doing food recall diaries.
Memory Based Food Recall is Terrible Part 2
In a detailed paper on the topic, they state the following
Unlike most research findings (7) the lack of credibility of M-BM [Memory-Based Methods] data has been replicated consistently over the past 3 decades. (3, 8, 9) We know of no other data collection tool (in any field of actual science) that demonstrates a significant decrement in performance each time it is used, yet that is precisely what was found in the Energetics study.(10)
Now, reference 10 is a paper by Arab et al. It gave the same subjects 8 dietary recalls over the span of 2 months. Some were in lab, some were online, I think some were mailed. Now, they were looking at the tolerability of people doing these things more than the numbers. Basically, will people do these damn things a bunch of times? The answer apparently is yes.
But when Archer and Pavela analyzed the data, they found something interesting, again quoting from the above reference
Over the administration of eight 24-hour recalls, Arab et al(10) reported a statistically significant decreasing trend for energy (−535 kcal/d; P<.001) that varied by macronutrient (protein: −22 g/d, −88 kcal/d; fat: −27 g/d, −243 kcal/d; carbohydrate: −54 g/d, −216 kcal/d)
So every time the people did the food recall for the day, they kept reporting lower and lower intakes. Even if the first report was accurate, the more that people do it, the worse they get. And there’s nothing to indicate that the first one was accurate.
Researchers in actual sciences will try to validate their methods, taking the same measurement twice a few days apart to see how close they are. If the method is good, it’s in the usually in high 90% range with relatively low variation. It has to be otherwise measuring before and after the study is useless.
In the case of epidemiology, relying on memory based recall, not only are the self-reports awful, they get worse with every use. Even by the same person.
But that’s not the only problem.
Everything is Multi-Factorial
I say quite frequently that the word multi-factorial should be tattooed on everybody’s eyelids. Because everything is. No single food or nutrient causes a disease or makes you more or less healthy. No single aspect of lifestyle causes health problems. It’s the combination of all of it.
It’s Always Multi-Factorial
Because here’s what epidemiological studies usually try to do: link some single aspect (occasionally more) to the outcome they are examining. So boom, food recall, eggs correlated with an increased risk of heart disease. The rest of the diet? Irrelevant so far as they are concerned. It was the eggs and only the eggs.
Yes, I know that studies attempt to separate these factors out to one degree or another. It doesn’t change anything.
And you can’t do that, conclude that a single aspect of an overall lifestyle causes a single outcome. Well you can but it’s a crap method of doing “science”. And that’s epidemiology.
In the context of diet, the fact is that you have to consider the whole diet. As a researcher named Michael Blaha wrote
it is impossible to disentangle the effect of one particular food or one macronutrient from the accompanying foods and macronutrients that characterize a typical dietary pattern.
Because not only do we eat a mixture of foods on a daily basis, certain food patterns tend to cluster. So people who eat a lot of fatty charred meats often have an overall bad diet. They don’t eat many fruits and vegetables. They usually eat a lot of refined carbohydrates. Maybe drink sugary sodas. Yet somehow one thing is to blame.
By the same token, people who eat leaner cuts of meat show common patterns. More fruits and vegetables, less refined carbohydrates, you get the idea. Their micronutrient intake is assuredly different and better as well. It all contributes.
No matter how hard you try, you can’t separate out a single
food from the overall pattern.