Friday, November 22, 2013

The nut study -- the good, the bad, and the ugly

We take rare pleasure in praising an Associated Press piece about a paper just out in the New England Journal of Medicine ("Association of Nut Consumption with Total and Cause-Specific Mortality", Bao et al., NEJM 2013 369: 2001-2011).  The paper reports a study showing that risk of death from all causes is reduced among study participants who frequently eat nuts and author Marilynn Marchione does a nice job pointing out the caveats (well, apart from the suggestion that nut eaters are less likely to die).

Why might nuts be protective of your health?  Boa et al. note that "Nuts are nutrient-dense foods that are rich in unsaturated fatty acids, fiber, vitamins, minerals, and many other bioactive substances, such as phenolic antioxidants and phytosterols."

Boa et al. used data from the longstanding Nurses Health Study and the Health Professionals Follow-up Study, looking at the dietary intake and disease history of a total of 119,000 people. Dietary intake was assessed with food-frequency questionnaires sent to study participants every 2 to 4 years.  And the primary endpoint was death from any cause. 
In two large prospective U.S. cohorts, we found a significant, dose-dependent inverse association between nut consumption and total mortality, after adjusting for potential confounders. As compared with participants who did not eat nuts, those who consumed nuts seven or more times per week had a 20% lower death rate. Inverse associations were observed for most major causes of death, including heart disease, cancer, and respiratory diseases. Results were similar for peanuts and tree nuts, and the inverse association persisted across all subgroups.
Other studies have reported similar findings.  Bao et al. suggest several alternative explanations for these findings, other than the apparent protective effects of nuts.  The findings could reflect "confounding by unmeasured or poorly measured variables", although the authors believe this is unlikely because so much data have been collected on these study participants that they feel they could control for most potential confounders.  Unknown confounders can't be controlled for, as they point out.

Reverse causality is another possible explanation for their findings -- unhealthy people might stop eating nuts, for example, rather than that people who don't eat nuts become unhealthy.  But the researchers excluded people from the study once they reported illness, in order to limit this potential problem.

Bao et al. note that because their study is observational, they haven't demonstrated cause and effect, but they also note that their results are consistent with many other reports of the health benefits of eating nuts.  Indeed, the authors are, to an usual but praiseworthy extent, responsibly circumspect in their reporting of their results.

So, the ground was well-laid for Marchione to write a responsible story in turn, and she did. Among other issues, she points out, for example,
Researchers don't know why nuts may boost health. It could be that their unsaturated fatty acids, minerals and other nutrients lower cholesterol and inflammation and reduce other problems, as earlier studies seemed to show.
Observational studies like this one can't prove cause and effect, only suggest a connection. Research on diets is especially tough, because it can be difficult to single out the effects of any one food.
People who eat more nuts may eat them on salads, for example, and some of the benefit may come from the leafy greens, said Dr. Robert Eckel, a University of Colorado cardiologist and former president of the American Heart Association.

Read more here:
.So, kudos to Marchione for pointing out that there are limitations to this kind of study.  The evidence seems to be building in favor of the protective role of nuts, and this study adds to the evidence, but it is not definitive.

Funder bias
Marchione notes rather in passing that one of the funders of the study was the International Tree Nut Council Nutrition Research and Education Foundation, but she says that they had no role in designing or reporting the results.  This, of course, raises the question of whether they were involved in analysis, but let's assume they were not.  Marchione was correct to mention this bit of information because studies funded by industry are much more likely to report findings in the funder's favor than are independently funded studies -- this is common enough that there's a Wikipedia entry for "funding bias".

Knowing that the Tree Nut people weren't involved in study design or reporting is not entirely reassuring about there being no conflict of interest.  A study published in PLoS Medicine in 2007 ("Relationship between Funding Source and Conclusion among Nutrition-Related Scientific Articles", Lesser et al.) reported the results of an examination of the relationship between sponsorship and conclusions of studies of the health effects of non-alcoholic beverages. 
206 articles were included in the study, of which 111 declared financial sponsorship. Of these, 22% had all industry funding, 47% had no industry funding, and 32% had mixed funding. Funding source was significantly related to conclusions when considering all article types (p = 0.037). For interventional studies, the proportion with unfavorable conclusions was 0% for all industry funding versus 37% for no industry funding (p = 0.009). The odds ratio of a favorable versus unfavorable conclusion was 7.61 (95% confidence interval 1.27 to 45.73), comparing articles with all industry funding to no industry funding.
The authors conclude that this has "potentially significant implications for public health."

Why this bias exists isn't clear, and there are probably multiple causes. Some of it is perhaps malfeasance or bad science, industry buying results, but it can't all be that. It's possible that when results are negative they just aren't published, and, ok, it can be argued that that's a clear sort of malfeasance. Or Lesser et al. suggest that perhaps industry funds only studies that they believe will favor their product.  But then the prior knowledge should be reported accurately and built into the testing (e.g., in a 'Bayesian' way), rather than pretending that one has a true 'null' hypothesis.  Or investigators, knowingly or not, propose studies that are likely to show that a sponsor's product is beneficial.

Lesser et al. also suggest that all investigators are biased somehow -- e.g., they want to support their favorite hypothesis -- but that financial conflict of interest is a separate animal.  "We contend that financial conflict of interest is qualitatively different, producing selective bias that acts consistently in one direction over time." This is certainly true in clinical trials funded by pharmaceutical companies.  Cosgrove and Wheeler, for example, go so far as to call the involvement of pharmaceuticals in drug testing corruption in a paper published this year in the Journal of Law, Medicine and Ethics.

The problem here is that the nut study might well have been devised, conducted and analyzed in exactly the same way without industry funding.   But it might well not have, and we have no way of knowing.  Again, Marchione was right to bring this up, but stating conflict of interest doesn't automatically prevent it.


Holly Dunsworth said...

Why not see nuts as neutral and the food they replace as bad, rather than nuts as boosters?

Ken Weiss said...

Oh, you relativist!

It's a good point, and unclear (to me) if they controlled for total consumption....whether there was anything detectable that could be called replacement behavior.

But, then, why not what was replaced as neutral and nuts as positive?

Anne Buchanan said...

Thank you, Holly. That's in fact another explanation that Marchione suggests, which I neglected to mention.

Anne Buchanan said...

Another issue with this and all studies of the health effects of single foods is that it's I will say impossible to measure those effects separate from the rest of the diet, exercise and so forth.

Jennifer said...

eat what you want and what tastes good to you, in moderation. All the studies on what's good and bad are going to change within 10 years anyway. I'm going to eat breakfast of bacon (without nitrates, of course) and oatmeal. Just cuz we're out of eggs (free range, of course)

Jim Wood said...

Anne, after I saw the press release declaring that eating nuts would lower your risk of dying, I immediately went and scarfed a handful of almonds. I interpret the wording to mean that, if nuts lower the relative risk of death by, say, 10%, then I have a 10% chance of being immortal, right? Right?? Pardon me, but I gotta go eat some walnuts.

Anne Buchanan said...

I believe nuts lower your RR of dying by 22%, Jim. And that's if you've eaten them daily for 30 years. If you eat them daily for 60 years, that must mean your risk of dying is now lowered by 44%? Unless it's a non-linear effect, in which case you are even further along the path to immortality. And I'm eating my almonds now. I'm right there behind you!

Jennifer said...

and you know my opinion of exercise!!!

Ken Weiss said...

Yes it is truly incredible that the author of the report didn't realize that this was the immortality pill! I have some trail mix here, if you feel you need a nibble....

Anne Buchanan said...

Walnuts on that oatmeal? Mm, carmelized walnuts prehaps.

DDeden said...

Hey, do squirrels live forever?

Anne Buchanan said...

Oh. This is kind of like telling a meteorologist to just look out the window.

Ken Weiss said...

Unfair and unanswerable question, until you get rid of hawks and cats! It would be like saying that someone killed in a car crash, whether eating nuts or not, didn't benefit from nut-immortality.

Anne Buchanan said...

Spoken like an ex-meteorologist, Ken. But, you have a point. Good save.

Jim Wood said...

There is a strain of thought among health policy mavens that all mortality that we've ever observed (with the possible exception of Madame Calment) was "premature" and that the goal of medical and public health measures ought to be to "rectangularize the survival function," i.e. to allow everyone to survive until their genetically-determined maximal life span (often assumed to be about 120 years), at which point they inexorably keel over dead. I always thought it would make the night before your 120th birthday party pretty bittersweet. Care for some peanuts?

Ken Weiss said...

This is almost what preachers promise (only that's eternal life). Not many years ago the leading lights of human genetics (you can guess their names) were promising that we would live as long as Methuselah, 400-800 years. Yes, these were scientists, not ad agencies (if there's a difference). Not exactly immorality, I guess.

Not that many years before that, your taxes were used (as you know, Jim) to prove the existence of a species-specific 'maximum lifespan potential' to which (give NIA enough funds!) we will all be able to live. That was the death-stroke age that you refer to. Why we would want to pay taxes only to be given our date-certain to worry about is anyone's guess. Perhaps it's so one's final 'going away' party won't be misrepresentative and let down all those in attendance.

Anonymous said...

Been seeing these studies since I was a young lad. This is good for you. That is bad for you. And so on.
The big problem I see is that it's impossible to control for every other factor, and there are so, so many.

Fact is, people who are healthier and more concerned about their health tend to eat more nuts.

What? They controlled for every other input? Smoking? Exercise level? High carb diet? Low card diet? Sugar input? Fiber input? Family history? Recreational drug use? Basal metabolic activity level?

Please. These studies are all flawed in the same way - they impliedly claim to restrict the analysis to a single variable, but, in reality, that's an impossible goal and the only reasonable conclusion that should be drawn is, "nut consumption is correlated with a healthy lifestyle."


Anne Buchanan said...

Yep. Or better, "nut consumption is correlated with a healthy lifestyle in *this* study." What you describe is the problem with all dietary studies, and indeed with environmental epidemiology in general. And indeed with genetic studies reporting the effects of single genes. And so forth.

Ken Weiss said...

Or better yet: there are fundamental flaws in how we are doing research, how it's being paid for and how much is being paid for it, and how it's being used and reported. Even obvious instances like this (and almost others, in the daily media) don't seem to have the traction to change how the research enterprise is doing its business.

John R. Vokey said...

It is a kind of Pascal's wager: Given that wealth is the greatest predictor of health, I am holding out for more money. Even if it turns out to be wrong, I at least can have all the other advantages available with wealth. If tree nuts turn out to be wrong...

S.H. said...

Thanks for this thoughtful post and for pointing to the funding source and associated issue of potential bias. As a researcher who studies PUFAs, I have always wondered about a little noted paradox: Nuts are now considered healthy despite their high content of omega-6 polyunsaturated fatty acids which leads to almost scary omega-6:omega-3 ratios (low ratios are good for you).

Now one could use the arguments in this post regarding study bias to reconcile this old paradox.

Yet, I do believe that there is validity in the finding of the beneficial effects of nuts. Bias due to financial interest usually concerns the suppression of a hoped-for absence of a health hazard (tobacco industry and smoking) or the hoped for presence of a plausible but questionable beneficial effect (vitamins!). The case of nuts is just neither. Their health benefit was rather unexpected. In fact, there is so much prejudice among my elderly patients against nuts that it is often impossible to convince them of the potential health benefits. Also, I heard that it was only after reports of the health benefits of nuts , notably almonds, started to accumulate that the almond industry association in California decided to launch their campaign to promote nut awareness. Just look at how nuts are sold (compared to vitamins): in rather neutral packing. Hence, the condition of expectation that I think should be met before on talks about bias, is not present.

Thus, in this case the link between source of funding and potential bias due to COI is, well, likely just correlation and not causation.

Anne Buchanan said...

Thanks for your comments. I agree with you that the link between source of funding and potential bias with respect to the nut study is likely to be correlation and not causation. But, to me, the fundamental problem with possible COI is that it raises questions about the credibility of a study, not that it *necessarily* means a study is biased, or worse. Can we trust these findings? Given that many epidemiological studies are already laden with potentially important biases, confounding, less than optimal statistical analysis and so forth, if you throw in potential COI that *could* have affected study design or analysis or reporting, one would be forgiven for concluding that all bets are off as to how to interpret study results.

Your comment about nuts and omega-6 polyunsaturated fatty acids is interesting. To me, it reflects a (the?) problem with nutrition studies that it's not clear how to get around -- we don't eat only a single dietary constituent, it's always in the context of a complex diet, everyone's unique genome, background and lifestyle. So, assessing the health effects of any single dietary constituent, as a long history of conflicting and non-replicable nutrition studies has shown, is problematic. This doesn't mean they have no effect, but if the effect is small, it's hard to identify.