The Drivers of the Herd, Part 9
Siri-Tarino’s Meta-Analysis, Part 1 (Saturated Fat and Heart Disease)
Slide 3 Siri-Tarino, Patty W., et al. "Meta-analysis of prospective cohort studies evaluating the association of saturated fat with cardiovascular disease." The American journal of clinical nutrition 91.3 (2010): 535-546.
This paper by Patty Siri-Tarino and her respected coauthors did a great disservice to the effort to reduce chronic disease worldwide. It’s used over and over now to convince people to adopt unhealthy diets. Few who refer to it understand its limitations. They don’t appreciate the superficial nature of this sort of statistical analysis, and they misconstrue the phrase “no significant evidence” to mean there is no meaningful evidence linking saturated fat to heart disease. “Significant” is here used in the statistical sense. This paper doesn’t even pretend to address most of the evidence in diet-heart.
Slide 4 http://www.abc.net.au/catalyst/heartofthematter/
Not only did the Catalyst program select this paper to bolster their forced and illogical attack on diet-heart, they even listed it twice on their site. They probably didn’t mean to do that. Their work is just very sloppy.
Slide 5 Freedman, David H. "Why scientific studies are so often wrong: The streetlight effect." Discover Magazine (2010): 26.
This meta-analysis by Siri-Tarino and company confined itself to a small subset of the literature on diet-heart. This portion of the literature lends itself well to statistical analysis. Prospective cohort studies are where the metaphorical light is on in diet-heart, and so that’s the only place they looked. In doing so they made no attempt at a qualitative analysis of the studies upon which their analysis was based. This video will demonstrate that to you and show you how a misleading impression can be created by statistical evidence.
Slide 6 This is the list of studies they used. Due to time constraints I have not been able to present to you commentary on them all.
Slide 7 Instead, I’ll only comment on some of them. Which ones? I’ll only be discussing those that are the most favorable to saturated fat. Here’s how this will work. The findings of this meta-analysis were presented in a form called a forest plot. The vertical line on the right represents no effect of saturated fat one way or the other. The area to the left of it implies protection from saturated fat and the area to the right represents increased risk from saturated fat. The horizontal lines represent the confidence intervals for each paper here. For our purposes, those lines don’t matter. I’ll be focusing instead on the dot at the center of each confidence interval line. To bias my review in favor of saturated fat, I’ll only be discussing with you those studies that center to the left of the vertical line. If those studies imply anything, they imply a benefit from saturated fat. This is my usual approach. I do not believe in dismissing the studies that support the other side. I think it is always better to try to understand them. I will not dismiss any of them with the usual copout, “correlation isn’t causation.” What you’ll understand by the end of this list is that while a meta-analysis based solely on reported data may be a worthwhile exercise when looking at drug trials, such an analysis fails on a subject which requires qualitative analysis as much as statistical analysis. Such a subject is diet-heart.
Slide 8 Stamler, Jeremiah. "Diet-heart: a problematic revisit." The American journal of clinical nutrition 91.3 (2010): 497-499.
Before I start, I must encourage you to read Jeremiah Stamler’s piece criticizing this meta-analysis. I will avoid repeating his criticisms, which isn’t easy since he responded so well. As far as I can tell, Dr. Stamler never received an appropriate reply to his criticisms from the authors of this meta-analysis.
Slide 9 Also, remember as we move along that this paper is used as evidence by the fat apologists when they argue that saturated fats are good for us. For each study you might ask yourself if saturated fat really did come off as innocent or even beneficial. You’ll see that the fat promoters will find almost nothing in any of these papers that sends a positive message about saturated fat.
Slide 10 Let’s get into this already. The very first study in favorable territory for saturated fat was authored by McGee.
Slide 11 McGEE, DANIEL L., et al. "TEN-YEAR INCIDENCE OF CORONARY HEART DISEASE IN THE HONOLULU HEART PROGRAM RELATIONSHIP TO NUTRIENT INTAKE." American journal of epidemiology 119.5 (1984): 667-676.
This one provides a great example of the inadequacy of the meta-analysis. This paper was based on the Honolulu Heart Study. We looked at that study in video 5. Read this abstract along with me.
“Men who developed coronary heart disease had a lower average intake of calories, carbohydrates, starch, and vegetable protein than men who remained free of coronary heart disease. Men who developed coronary heart disease also had a higher mean intake of percentage of calories from protein, fat, saturated fatty acids, and polyunsaturated fatty acids than men who remained free of coronary heart disease. These men also had a significantly lower mean percentage of calories from carbohydrates and a higher mean ingestion of cholesterol per 1000 calories than men who remained free of coronary heart disease.”
That’s all pretty clear, is it not? More saturated fat, more meat, more cholesterol, more heart disease. How on earth did they use this to say that there is no significant evidence that saturated fat causes heart disease? Well, if you read on, you see that they adjusted their data for serum cholesterol. This is rigging the game because saturated fat and dietary cholesterol raise blood cholesterol. If you minimize that effect by adjusting for cholesterol, you’ve just removed any connection between your analysis and the real world.
Slide 12 Stamler, Jeremiah. "Diet-heart: a problematic revisit." The American journal of clinical nutrition 91.3 (2010): 497-499.
Jeremiah Stamler pegged this one. It’s an example of overadjustment. There is no doubt that every single author of this meta-analysis knows this would have been a problem and yet they used this study this way anyway.
Slide 13 McGee, Daniel L., et al. "TEN-YEAR INCIDENCE OF CORONARY HEART DISEASE IN THE HONOLULU HEART PROGRAM RELATIONSHIP TO NUTRIENT INTAKE." American journal of epidemiology 119.5 (1984): 667-676.
Read what the paper actually said. “Serum cholesterol strongly retained a significant association with total coronary heart disease and nonfatal myocardial infarction.” Thanks to Patty Siri-Tarino and company, this study is now used to argue exactly in opposition to this statement.
Slide 14 The next study we’ll consider is by Posner.
Slide 15 Posner, Barbara Millen, et al. "Dietary lipid predictors of coronary heart disease in men: the Framingham Study." Archives of Internal Medicine 151.6 (1991): 1181.
There you see the abstract. Saturated fats had a marginally significant positive association with heart disease but that was about it. This was based on a single interview of each participant at the beginning of the study covering the previous 24 hours. They followed up after 16 years.
Slide 16 Here you see that this was based on an assessment, not multiple assessments. This study started in the late ‘60s. Don’t all the low-carbers say that America adopted a low-fat diet during the ‘70s? Why would we assume that their diets stayed the same for 16 years, the same as they were on that one day in the 1960s? This is a major weakness of this paper.
Slide 17 This study had the usual problems associated with studies done within homogeneous populations. Look how high their saturated fat consumption was. It was around 15% of calories. Their dietary cholesterol was well over 500 mg. That number should be zero in my opinion.
Slide 18 http://health.gov/dietaryguidelines/dga2010/dietaryguidelines2010.pdf
In the opinion of the USDA saturated fat should be under 10% and cholesterol should be under 300 mg. Even by those loose standards they were way too high.
Slide 19 Consequently, as a population, they had high cholesterol.
Slide 20 With saturated fat and cholesterol consumption that high, the researchers decided to look for what was happening with the few people who were eating reasonably healthy diets. Here’s a line that was lost on the Siri-Tarino paper:
“Those who consumed 10% of energy intake from saturated fatty acid, compared with the mean level of 15.2%, had a CHD relative risk of 0.78…”
That means that those who were near our currently recommended limit for saturated fat – the people who consumed the least saturated fat in this cohort – had a much lower risk of suffering from heart disease. Again, how does a study like this get twisted to support saturated fat consumption? Whatever the reason, it sure wasn’t to benefit you.
Slide 21 You may have noticed in the abstract that they found that total fat and more importantly, monounsaturated fat had a significant positive association with heart disease among the younger people.
Slide 22 They pointed out here that in this group, that monounsaturated fat wasn’t coming from vegetable sources like olive oil. In this cohort that fat was coming from animal products, and it didn’t seem so healthy.
Slide 23 National Institutes of Health. "Third Report of the National Cholesterol Education Program Expert Panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III)." NIH publication 1 (2001): 3670.
Before we leave this paper about Framingham, I’ll just use these graphs to remind you how strongly total cholesterol associated with risk of heart disease in Framingham. That’s where this data came from. Don’t let anyone tell you different about Framingham.
Slide 24 Next in line is Goldbourt. We became familiar with this one during that Catalyst video, too. This will be a quick review.
Slide 25 Goldbourt, U., S. Yaari, and J. H. Medalie. "Factors predictive of long-term coronary heart disease mortality among 10,059 male Israeli civil servants and municipal employees. A 23-year mortality follow-up in the Israeli Ischemic Heart Disease Study." Cardiology 82.2-3 (1993): 100.
In this cohort, a direct relationship was found between fatal coronary heart disease and cholesterol. Cholesterol levels are the main reason we care about saturated fat, so this is not a study a cholesterol apologist should be citing.
Slide 26 Here you see this stated another way. High cholesterol gave a “clear disadvantage.”
Slide 27 Once again, notice the countries with the highest and lowest prevalence and incidence rates. Men from Central and Eastern Europe had the highest prevalence.
Slide 28 Kahn, H. A., et al. "Serum cholesterol: its distribution and association with dietary and other variables in a survey of 10,000 men." Israel journal of medical sciences 5.6 (1969): 1117.
They also had the highest saturated fat consumption, as determined in another paper based on this cohort.
Slide 29 Those with the lowest prevalence and incidence were from Middle Eastern and North African countries.
Slide 30 Kahn, H. A., et al. "Serum cholesterol: its distribution and association with dietary and other variables in a survey of 10,000 men." Israel journal of medical sciences 5.6 (1969): 1117.
And those were the men who consumed the least saturated fat.
Slide 31 Goldbourt, U., S. Yaari, and J. H. Medalie. "Factors predictive of long-term coronary heart disease mortality among 10,059 male Israeli civil servants and municipal employees. A 23-year mortality follow-up in the Israeli Ischemic Heart Disease Study." Cardiology 82.2-3 (1993): 100.
The paper Siri-Tarino and company considered had some of the usual weaknesses. They used a single estimate from all the way back in 1963 for dietary intake. That information was supposed to represent the diets of these guys over the next 23 years. The authors knew this was a weakness here.
Slide 32 Let’s look at the table showing fat intake compared to incidence of heart disease. You can see that as saturated fat consumption went up in absolute terms, heart disease went down.
Slide 33 But you can also see on the left that as saturated fat consumption went up as a percentage of fat, coronary heart disease went up, too. That trend relates to the column on the right, showing that these guys were better off the more they traded saturated fats for polyunsaturated fats. So you can see that one can easily interpret this study as affirming the lipid hypothesis.
Slide 34 Here you can read their description of what we just saw. When saturated fat was assessed as a percentage and as a ratio, its association with death from coronary heart disease was significant. Again, this didn’t look like a particularly supportive study for saturated fat.
Slide 35 We skip a few that clearly showed harm from saturated fat and some that didn’t say much either way. Again, I was limited in the time I could devote to these videos. Therefore, we’ll skip to the next favorable study. The author is Pietinen.
Slide 36 Pietinen, Pirjo, et al. "Intake of fatty acids and risk of coronary heart disease in a cohort of Finnish men: the Alpha-Tocopherol, Beta-Carotene Cancer Prevention Study." American Journal of Epidemiology 145.10 (1997): 876-887.
Notice in the middle of this excerpt from the abstract that omega-3 fats from fish were associated positively with coronary death. That’s not what we’re usually told. Anyway, if you look at the second to last sentence you’ll see that saturated fat intake was not associated with coronary death, nor was dietary cholesterol. Only trans fats came off poorly here.
Slide 37 This was a six-year [study]. Diet was only assessed at the beginning. This is a typical shortcoming.
Slide 38 Unsurprisingly, serum cholesterol was directly associated with coronary events and coronary death. Once again, we see that when we look under the hood of a pro-fat paper, we see that it isn’t all that pro-fat.
At this point you are probably wondering why a study would simultaneously find that cholesterol associates with mortality but saturated fat does not. I wondered that, too.
Slide 39 Notice a couple things here. First, look at the bottom. This was a very high-cholesterol cohort. That figure at the bottom converts to about 236 mg/dL for us here in the United States. We’re again looking at a population lacking in meaningful variations in lifestyle so it would be hard to tease out good information here. But that’s a very minor concern. Notice the top sentence. “Intakes of saturated fatty acids and cholesterol were lowest in men in the lowest and highest quintile of trans-fatty acid intake.” In other words, the men who were eating less artery-damaging saturated fat were eating more artery-damaging trans fat. Great…
Slide 40 There you see our problem. Those with the greatest intake of trans fats, more than twice that of the next highest group, also had the least intake of saturated fats. And that, my friends, is what you call “confounding”. Even I won’t tell you that saturated fats are worse than trans fats. They’re both just terrible for you. Beside that, notice on the right that across the spectrum of trans fat intake, everyone was eating quite a lot of cholesterol. As usual, you can be sure that hardly anyone in this bunch was consuming a true low-fat diet based on whole plant foods. All the men in this study were smokers, too.
Slide 41 Now it should be clear to you that this was not a study looking at the effects of saturated fat. This was a study that compared saturate fat with trans fats. Unsurprisingly, the results were a wash. You just had a bunch of old guys eating terribly and dying fast.
Now that you’ve seen that, ask yourself if Patty Siri-Tarino and her well-known colleagues don’t know that trans fat is bad for your heart. Did they even read that one?
Slide 42 A study by Oh is next on our list.
Slide 43 Oh, Kyungwon, et al. "Dietary fat intake and risk of coronary heart disease in women: 20 years of follow-up of the nurses' health study." American Journal of Epidemiology 161.7 (2005): 672-679.
This one is pretty crazy. Here’s what these authors said: “Intakes of saturated fat and monounsaturated fat were not statistically significant predictors of CHD when adjusted for nondietary and dietary risk factors.”
Notice they are not saying that saturated fat consumption wasn’t a predictor of heart disease. They are saying that it wasn’t after they made their adjustments.
Slide 44 Here were their adjustments. The first model adjusts for dietary cholesterol. That’s removing an important contributor to blood cholesterol and therefore heart disease risk, but let’s let that slide. Notice the symbol for the next adjustment. I’ll confess I don’t understand what’s going on here. That symbol means they adjusted for all the previous variables plus some more, including saturated fat. Now you would think that surely they are not saying that saturated fat consumption showed no association after they adjusted for saturated fat consumption. That would make no sense at all. Now look carefully at that second symbol. It’s a cross with an extra slash.
Slide 45 There is that symbol. They are applying it to saturated fat intake. I don’t understand this, folks. If you know what this is about please contact me. With that adjustment, you see that there wasn’t any trend as we go up through the categories of the risk of heart disease in this cohort.
Slide 46 But here’s the indication something very strange is going on. Both the unadjusted and age-adjusted associations show very clear trends. To me, this data says that low risk meant low saturated fat consumption and high risk meant high saturated fat consumption. Hit pause on this video, study this table, and make up your own mind.
Slide 47 There was one other complicating factor here. This cohort of nurses reported a rather dramatic decrease in their consumption of saturated fat over this time period. The authors said this may have weakened any relationships between saturated fat and heart disease.
Slide 48 But these numbers here are speaking pretty loudly to me. Saturated fat is looking rather guilty in this one.
Slide 49 Our last study in the coronary heart disease category is by Leosdottir.
Slide 50 Leosdottir, Margret, et al. "Cardiovascular event risk in relation to dietary fat intake in middle-aged individuals: data from The Malmö Diet and Cancer Study." European Journal of Cardiovascular Prevention & Rehabilitation 14.5 (2007): 701-706.
This one reported that they didn’t find any benefit in their data for the restriction of saturated fat intake.
Slide 51 I need to show you some tables so you can see what’s going on here. Those columns labeled 1st through 4th are for quartiles of fat intake. It would be better if they were for quartiles of saturated fat intake instead but we’ll have to make do with what they gave us. Fortunately, you can see that as fat increased so did percentage of saturated fat, whereas the percentage of polyunsaturated fat didn’t move very much. This data will do for our purposes. You see they made adjustments. These adjustments are what messed up this study. Adjusting for smoking, for example, means that they applied a distortion to their raw data so that those who smoke more are counted less when they are looking at the effects of saturated fat. Watch how this works.
Slide 52 There are the percentages of smokers according to quartiles of fat intake. As you can see, smoking increases right along with fat and saturated fat intake. How did they adjust for smoking without losing the effect of saturated fat? They probably didn’t.
Slide 53 They also adjusted for fiber intake. As fat consumption rose, fiber consumption fell. Again, adjusting for fiber would have distorted the relationships to minimize the effect of saturated fat since there was so much consistency in this trend. This study looks like another example of overadjustment, just as Dr. Stamler said.
Slide 54 Lastly, I included this line under their table so you can see they also made adjustments for the polyunsaturated to saturated fat ratio. This is key since polyunsaturated fats generally reduce cholesterol and therefore heart disease risk in populations that consume too much fat. Once again, their adjustments got in the way of the real relationships. In effect, they are saying “trust us.” I for one am doubtful of their methods.
Slide 55 Let’s pause for a moment. Notice that we are at the end of the list of studies that looked just at saturated fat’s effect on the risk of coronary heart disease. That diamond you see across from the subtotal tells us where everything averages out. Realize we skipped studies that showed very clear associations between saturated fat and heart disease. I trust you’ll agree that even the most favorable studies made saturated fat look pretty guilty. The evidence connecting saturated fat consumption to heart disease is very robust. But you wouldn’t get a sense of that from that diamond. We should be able to agree on this before we move on to the relationship between saturated fat and stroke. It’s only when we look at stroke is there a reasonable argument to be made in favor of saturated fat. Yes, there is some evidence here that argues for saturated fat consumption. And you’ll see I won’t shy away from it in the next video.