When the methodology fails

An op-ed piece by Nina Teicholz in Friday's NYTimes lays it on the line, chastising the government for its regular bulletins on dietary advice that, for 50 or so years have altered what we eat, what we fear to eat, and what the risks are.  Now, new studies tell us that what was bad is good and what was good is bad, and that the prior half-century of studies were wrong.  We've eliminated fats and cholesterol, and replaced them with carbohydrates, but, as Teicholz writes,

...recent science has increasingly shown that a high-carb diet rich in sugar and refined grains increases the risk of obesity, diabetes and heart disease — much more so than a diet high in fat and cholesterol.

But why should we believe these new studies?  Teicholz basically takes the underlying methodology to task, and yet she has written a book recommending that we eat more fats (“The Big Fat Surprise: Why Butter, Meat and Cheese Belong in a Healthy Diet"), but those recommendations are based on the very same faulty methodology as the recommendations with which she, and the current USDA advisory committee, find fault.

Embrace the fat! (Wikipedia)

The same, almost exactly the same, critiques are earned by many of the 'big data' genomics studies (and other long-term go-not-very-far megaprojects).  It is the statistical correlation methodology.  When many factors are studied at once (perhaps properly since many factors, genetic and environmental, are responsible for health or other traits), we can't expect simple answers.  We can't expect correlation to imply causation.  We can't expect replication.  We can't predict the risk factors that people, for whom risk advice is based on such studies, will face in the future.

The real conclusion is to shut down the nutrition megaprojects at Harvard (singled out by the op-ed) and the other genetics and public health departments that have been running them for decades, and do something different.  The megaprojects have become part of the entrenched System, with little or no real accountability.

Pulling the plug would be a major acknowledgment of failure, both by the feds for what they funded, the program officers for defending weak portfolios and their budgets, the universities defending their overhead and prestige projects and, of course, the investigators who are either simply unable to recognize what they're doing, or too dishonest and self-protecting to come clean about it.  And then they and their students could go on to do something actually productive.

Of course such a multi-million dollar threat will be resisted, and that's why the usual answer to the kinds of conflicting, confusing reports that so often come out of these megaprojects is to increase their size, length and, geez, what a surprise!, their cost.  To keep funding the same investigators and their protegés.  This is only to be expected, and many people's jobs are covered by the relevant grants, a genuine concern.  However, research projects are not supposed to be part of a welfare system, but to solve real problems.  And the same peoples' skills could be put to better use, addressing real problems in ways that might be more effective and accountable.

And we used to laugh at the Soviets' entrenched, never successful, Five Year Plans!

It is a public misappropriation that is taking place.  Yes, there are health problems we wish to avoid, and government and universities are set up to identify them and recommend changes.  But, for most of today's common chronic diseases, lifestyle changes would largely do the trick.

But then, that would just let people live longer to get diseases that might be worse, even if at older ages.  And meanwhile we aren't putting on a full court press for things that really are genetic, or really do have identifiable life-style causes.

Much of this research is being done at taxpayer expense.  We should let the people keep their money, or we should spend it more effectively.  We won't be able to do the latter until we admit, formally and fully, that we have a problem.  Given vested and entrenched interests, getting that to happen is a very hard trick to pull off.

What's a 'healthy diet' anyway?

A nutrition advisory panel is convened by the US Department of Agriculture every five years to review recent research in the field, make sense of it, and offer recommendations about what Americans should be eating for optimal health.  Those food pyramids, MyPlate, decades of advice to limit our cholesterol, saturated fat and salt intake?  All the work of these panels of experts who scoured the data and told us what it meant.  And as a result, from the 1960's onward, good, conscientious people reduced their cholesterol intake, took the salt shaker off the table and the whole milk and butter out of the refrigerator, cattle were bred to be leaner, eggs were banned from breakfast, low-salt and low-fat processed foods appeared on the shelves, and heart disease death rates ... continued to fall.

Deaths due to diseases of the heart (United States: 1900–2006). Circulation.2010; 121: e46-e215

Now, according to numerous news accounts, including here at the Washington Post, after more than 50 years of anti-cholesterol, anti-fat expert advice, the current advisory panel is reportedly poised to recommend that we need no longer need to limit the amount of cholesterol we eat, nor worry about our salt intake.  Bring on the shrimp, the lobster, the eggs, the meat, banish the guilt!

Oh, wait.  Hold on a sec.  Let's go back to those falling heart disease rates.  I remember one of the first lectures I heard as a new grad student at the University of Texas School of Public Health in the late 1970s, given by heart disease epidemiologist Reuel Stallones, Dean of the school.  His point was that rates had been falling since the 1960's and epidemiologists had no idea why.  He systematically destroyed every argument then (and now) current that might explain the rise and fall of heart disease death rates -- changing diet, decreased smoking, de/increased exercise.

In fact, he made the same case in 1980 in Scientific American in an article called "The Rise and Fall of Ischemic Heart Disease".  (The terms ischemic heart disease, coronary heart disease, and arteriosclerotic heart disease are more or less interchangeable, according to Stallones.)

In the U.S. the death rates attributed to heart attack and other results of the obstruction of the arteries that nourish the heart have fallen since the 1960's. Why they have is not understood.

Here's Stallones' graph of heart disease death rates, from 1900 to 1980.  I can't explain why deaths are still rising in the above graph in the 1950's, but falling in this graph; presumably this has to do with differing classifications of deaths due to heart disease.  Anyway, here rates rose rapidly starting in 1920 and then began to fall in the 1950's, steeply in the 1960's.

Stallones, Scientific American 354(5); 53-59

Here's another graph showing the decline more or less in line with Stallones' data, from a 2000 paper in Circulation.

"Death rates for major cardiovascular diseases in the United States from 1900 to 1997. *Rates are age-adjusted to 2000 standard." Source: Circulation, Cooper et al., 2000

As Stallones wrote, "Plainly a sustained decline in the death rate for ischemic heart disease commands attention and calls for explanation." And, he backs up to ask not only why the fall, but why it was that heart disease mortality began to rise so quickly in 1920, particularly among men.  Whatever explains the decrease must also explain the increase.

Smoking began to rise after World War 1, which fits the rise in heart disease mortality, and began to decrease from the mid-1960's or so, which fits the decrease.  But such an explanation would assume no latency period between beginning to smoke and its effects on heart disease.  And, middle-aged men quit smoking at higher rates than women, and this is not, as Stallones said, in concordance with the pattern of decline in heart disease mortality.  Treatment isn't an effective explanation either, because incidence rates -- new cases -- followed the same pattern as mortality.

He concluded,

In summary, four major variables are known to be associated with the risk of ischemic heart disease in individuals. Among the four, hypertension does not fit the trend of the mortality from ische­mic heart disease at all; physical activity fits only the rising curve, serum choles­terol fits only the falling curve and only cigarette smoking fits both. In no case is the fit as precise as one would like. This raises doubt that any of the factors is a fully satisfactory explanation for the variation in mortality.

So, in sum, as of 1980 epidemiology had no explanation for the rise and fall of heart disease mortality rates.

And epidemiology still can't tell us what causes heart disease, or predict who'll get it.  So, apparently we'll soon be told that we no longer have to monitor our cholesterol intake, and there's a lot of talk about fat consumption not being linked with heart disease anymore, and it's not clear whether obesity or hypertension are actually causal.  At least smoking is probably still a problem.

It's clear from the rise and fall of death rates through the 20th century that genes aren't going to be the major explanation because genes can't explain the spike in the 1920's and the fall 40 years later.  That experience also makes it clear that we can't predict environmental changes (or, often, even figure out what they were in hindsight) that might be associated with risk, and thus we aren't going to be able to predict the future, despite the claims of precision medicine advocates.

Stallones suggested that heart disease mortality data might indicate a single environmental cause to explain the rise and fall of death rates, but found reasons to argue against each of the most obvious ones.  Could the cause have been inflammatory?  If so, that would reinforce the idea that predicting future environments and causes is not going to be possible.

And, if there was a single cause, it's curious that our reductionist approaches, with large carefully designed samples and sophisticated statistical analysis, were unable to identify it, because that's what they are widely thought to be best at.  This makes it more likely that heart disease in populations has multiple causes.  And in fact every heart attack is unique, because no two people eat the same things, do the same amount of exercise, suffer the same infectious diseases, and so on. So maybe the very word 'cause', and the very approach (statistical), both of which assume some regular, replicability properties, are not being appropriately conceived.  This is a subject we'll discuss next time.....

"Experts" responding to the coming cholesterol recommendations have said that we still need to eat a healthy diet.  But when we still have no idea what's unhealthy, it's hard to know what is.

Common ground

By Eric Sannerud

Note:  We know Eric through our connection with a remarkable group of farmers, philosophers, economists, geneticists, innovators, writers, both academic and not, who share a concern for how we humans are mismanaging our place in the biosphere, and how we might make it better.  Eric describes himself as a farmer, thinker, and entrepreneur in Ham Lake, Minnesota. He is the Director of Sandbox Center for Regenerative Entrepreneurship and a member of the Minneapolis Hub of the Global Shapers. Connect with him on Twitter @ericsannerud.   Here are his thoughts:

As a 23 year-old American farmer who studies the US food system from the field I have a unique perspective on the serious challenges it faces. From drainage tiles that evacuate nutrient laden water to the nearest public water source, to obesity rates that cost untold lives, livelihoods, and money, the US food system is badly in need of regeneration.

Source: Wikipedia

Food and health policy in the United States.
In the United States food policy is a collection of local and national priorities that concern the supply of food. US food policy sets supports for certain crops that lead to a higher supply (and therefore lower price) of these crops in the market. Crops that are insured by the US government, against too much rain or too much drought, for example, such as corn, soy, and wheat, are more attractive to farmers than “non-insurable” crops, leading to greater production of insured crops.

United States Health policy is a collection of state and national regulations meant to minimize occupational and recreational dangers and to improve health. Seat belts, MyPlate.gov, the newest iteration of the government recommended diet, FDA regulations, and food labeling mandates are examples of health policy. The intended purpose of many of these regulations, as they relate to food, is to educate consumers to make informed decisions about what they eat. MyPlate identifies proper serving sizes for Americans (though it is not without criticism1). Food labels provide even, consistent criteria for comparing two different items (even if less than half2 of Americans read them).

The trouble is this...
On one hand we have food policies, such as government crop insurance, that encourage environmentally damaging fence row to fence row crop production, or government support for drain tile, drainage systems for fields that shuttle nutrient rich runoff to the nearest water body to be rushed downstream. On the other hand are well-meaning health policies. One can imagine that in the minds of the crafters of health policy each consumer carefully reads the food label on each product, compares the serving size of their meals against the MyPlate recommendations, and eats just the right amount of calories for their BMI each day. Real life food decisions are more complex and price is a big factor in purchasing. Price is where food policy gets involved. Government support makes certain crops cheap. These cheap crops can be used to create cheap food products (corn into chips and soda, for example). But chips and soda are shunned by health policy, and do not have healthy nutrition labels or a formal home on MyPlate.

Due to this disconnect between food policy and health policy the US food system is malfunctioning. A food system that creates historic rates of obesity3 while continuously exploiting the resources humans require for life, soil4 and water5, requires change. However, since the left hand doesn’t know what the right hand is doing, any efforts made, positive or negative, will be hindered by inefficiencies and ineffectiveness.

There is no one elegant solution to reducing the negative effects of such disconnected policies. Anyone claiming to have a trump card is lying: GMOs will not solve all of our problems, neither will organic production nor sin taxes on fizzy drinks and new government serving size suggestions. When dealing with interconnected systems solutions require a full deck of answers.

Three cards to add to the deck

1. Regenerating Health
US consumers shop with their wallets while health policy targets their minds. Health policy that acts on this fact will be moving in the right direction. The question for the discerning health policy strategist then is how to make healthy food price competitive?

One argument that I find persuasive as a low-paid, full-time change maker is the prudence of home cooking. Too often on the run I need food that is grab and go. Frozen burritos at the store cost me $2.00 each, I can make similar quality, though I must say, far tastier, burritos at home for just $.75.

A more aggressive strategy than home cooking promotion is artificially adding cost to unhealthy food. The reasoning goes that if that 76oz soda costs $5.00 instead of $1.00 less people will imbibe. Unfortunately, according to a recent US Supreme Court ruling all Americans have the right to drink cheap soda.6

One inventive way that communities across the United States are improving the cost competitiveness of healthy food is by offering “bonus bucks” to Electronic Benefits Transfer (EBT), government food support, purchases. Spend $20 of EBT at a participating farmers market and get $5 additional “market bucks” good for any fresh produce at the market.

2. Regenerating Land
More healthy food in the market will make healthy food cheaper and more accessible. A benefit of coordinated food and health policy is an increase in the overall supply of healthy food.

Farm fields: Wikipedia

For starters, imagine if US food policy aligned what farmers were incentivized to grow with what health policy encourages Americans to consume. The landscapes of rural America, and the tables of all Americans, could change drastically. This map7, by Emily Cassidy at the University of Minnesota’s Institute on the Environment, shows the caloric efficiency of crop production across the world. Caloric efficiency is the ratio of calories produced on a landscape to the number of produced calories consumed directly by humans. Developed countries producing commodities show horrendous caloric efficiency. Globally just 41% of calories produced are consumed by humans. According to Cassidy, maximizing caloric efficiency could feed an additional 4 billion people. In the US, food policy structures that support big commodity production could be amended to support crops, meat, and production methods of higher caloric efficiency including growing more crops for direct human consumption and more caloric efficient animal proteins such as chicken and fish.

Photo by Eric Sannerud

Private actors have their own part to play in addressing this disconnect. Non-governmental actors can work to aggregate and add scale to local food systems: decreasing prices of the freshest produce by harvesting efficiencies of scale.

Photo by Eric Sannerud

Two up and coming projects, Urban Oasis in St.Paul, Minnesota and New Moran in Burlington, Vermont, are examples of private sector innovation. By serving as aggregation, processing, and distribution hubs for local farmers these projects can increase the scale of healthy local food systems.

3. Regenerating Governance
Solutions also exist in state and local governments that can induce dialogue between government food and health policy makers.

At the state level food policy councils are popping up across the nation. These food policy councils are often created with the express purpose of increasing dialogue between state departments of agriculture, natural resources, and health. The Iowa Food Systems Council is one of the most longstanding and studied State food councils.

City level food councils are also developing. Similar to the state level councils these organizations are made up of a diverse group of stakeholders from across the food system including farmers, nutritionists, academics, and entrepreneurs. In Minneapolis, Minnesota “Minneapolis Homegrown” is a food policy council made up of appointed community members who serve an advisory role to the elected city council on food and health policies.

Hand, eye coordination

Food and health policies in which the left hand doesn’t know what the right is doing are only effective at continuing the failing status quo. At their best, food policy attempts to tackle resource issues in food production while health policy encourages healthy grocery store purchases. Discontinuity contributes to the symptomatic nature of present day solutions and thinking. A focus on symptomatic solutions leaves the underlying disease untouched. In order to cure the cause the US needs a new coordination between food and health policy. Thankfully, there are many luminaries across public, private, and government sectors who understand the underlying problem and are generating bold ideas to address it. 

1 The Nutrition Source. (2014): Healthy Eating Plate vs. USDA’s MyPlate. Harvard School of Public Health [online]. -URL: http://www.hsph.harvard.edu/nutritionsource/healthy-eating-plate-vs-usda-myplate/

2 The NPD Group. (2014, Feb 27): U.S. Consumers’ Interest in Reading Nutrition Facts Labels Wanes as Time Goes On, Reports NPD. NPD Group [online]. -URL: https://www.npd.com/wps/portal/npd/us/news/press-releases/u-s-consumers-interest-in-reading-nutrition-facts-labels-wanes-as-time-goes-on-reports-npd/

3 Centers for Disease Control and Prevention. (2012): Overweight and Obesity, Centers for Disease Control and Prevention [online]. - URL: http://www.cdc.gov/obesity/data/adult.html

4 Lang, S. (2006, Mar 20): ‘Slow, insidious’ soil erosion threatens human health and welfare as well as the environment, Cornell study asserts, by Cornell University [online]. - URL: http://www.news.cornell.edu/stories/2006/03/slow-insidious-soil-erosion-threatens-human-health-and-welfare

5 Bielle, D. (2008, Mar 14): Fertilizer Runoff Overwhelms Streams and Rivers, in Scientific American [online]. - URL: http://www.scientificamerican.com/article/fertilizer-runoff-overwhelms-streams/

6 Klepper, D. (2014, Jun 26): Drink Up NYC: Ban on Big Sodas Canned, in ABC News [online]. -URL: http://abcnews.go.com/Health/wireStory/court-reinstate-york-citys-big-soda-ban-24314227

7 Cassidy, E. (2013): Hotspots of inefficiency Mapping the difference between crop production and food calorie delivery. Institute on the Environment [online]. -URL: http://gli.environment.umn.edu/wp-content/uploads/2013/04/EmilyCassidy_AGU_small.pdf

Eat, drink, and be .... confused!

Is beer good for your health or is it a slow killer?  What about, say, bread, broccoli, wine, eggs, or (dare we say it?), sex?

Beer; Yebisu Beer Museum, Wikipedia

The answers are:
1.  Yes!  Anything we consume will keep us alive, but that just facilitates the path to our final end.  Metabolism and conjugal energy expenditure generate waste products, heat, cell division, and so on.  That leads to death!

Or 2.  Nobody knows!  After decades of huge, expensive studies by presumably the most knowledgeable investigators doing state-of-the-art science, we know relatively little with relatively little firmness, what we eat does for or to us.

In a Times article on Sunday, nutrition journalist Gary Taubes excoriates the nutrition research mill, and we think properly so, for decades of generating ever more numerous studies on nutritional epidemiology, without garnering many firm or important conclusions.

The 600,000 articles — along with several tens of thousands of diet books — are the noise generated by a dysfunctional research establishment. Because the nutrition research community has failed to establish reliable, unambiguous knowledge about the environmental triggers of obesity and diabetes, it has opened the door to a diversity of opinions on the subject, of hypotheses about cause, cure and prevention, many of which cannot be refuted by the existing evidence. Everyone has a theory. The evidence doesn’t exist to say unequivocally who’s wrong.

Even the basic questions about foods and eating are not being answered with much rigor, including even the common-wisdom recommendations related to obesity, dietary abuses, and so on, are often on quite shaky ground.  It is a huge research establishment that has its hands on funding agencies and is not being held accountable for delivering actual goods commensurate with the public investment.

Why can't we figure out the relationship of nutrition to disease in so many cases?  Largely because we've got a reductionist science that is able to find causes of disease that have large effects -- smoking, asbestos, the cause of infectious diseases -- but lousy at explaining diseases that are due to gradual exposure to multiple interacting factors that takes place over decades.  And, when some people exposed to what looks like a risk factor -- obesity, say -- develop diabetes but others don't, or when lots of sugar in the diet seems to be associated with obesity in some people but not in others, our methods really fail us.  We see the exact parallel in genetics, as we've written many times.  Indeed, does DDT cause Alzheimer's disease, a result we blogged about just last week?

Also on Sunday, the BBC reported that vitamin C is an effective treatment for cancer.  How many times for how many decades do we still have to hear more of this-finally-is true conjecture about vitamin C?  It's been going on for many decades.  The current story appeared in a journal called Science Translational Medicine.  The very phrase' translational medicine' reflects our rather bourgeois industrialization of the research system with its business and status basis.  It is a cachet self-congratulatory catch-phrase that suggests that biomedical research in the past had no interest in preventing or combating disease.  It suggests that grants were not given by NIH for such purposes (of course, NIH does fund a lot research that's irrelevant to health), which is just plain silly.  Or, rather, as we often have suggested, an establishment's typical way of making itself sound salubrious to the taxpayer we milk for our careers.  So why is there even a Science Translational Medicine journal?  Because Nature has one?  Because there might be advertising or subscription gains to be made?  Because each science publisher has to keep up with the perceived proverbial Joneses?  Because the bloated professoriate clamors for ever-more status-sounding places to publish their work?

What we're doing in the biomedical and health research establishment is to a great extent ever more of the same kinds of studies only bigger and with more costly and sophisticated hard and software.  Anyone with a computer can get SurveyMonkey software and design a questionnaire, and if you've got a degree in public health you know how to hire a bunch of nurse-interviewers, phone-callers, data-base miners and the like, and send them out on the streets to do various sorts of random samples, test and quality-check questionnaires in a standard way, then increase samples, have lots of meetings and data-enterers, and after a few years start using push-button statistical software to pour out papers (and contact the Times and BBC 'science' journalists to trumpet your work).  And, every year or so, write new grants to follow up your important 'translational' research.

Is there anything new here?
The  state of play is well known, and well known at least to the thoughtful contingent of researchers in the game.  Taubes, who has been guilty of simplistic advocacy as even he acknowledges by confessing his personal preference for sugar as the one-size-fits-all evil, clearly identifies the nature of the problem.  He doesn't really offer a solution.

We can't either, but we do say, that what is not being done is making better use of our wet ware: our brainpower.  Taubes' story was on nutrition research, and we have been harping on the same issues with respect to genomics, and have also critiqued epidemiology (including nutritional epidemiology) in past posts.

There is no magic answer for ginning up real insight and creativity.  But there may be ways of numbing or mesmerizing the part of society that might produce creativity.  A huge factory-like establishment of drone workers who need the factory to keep spewing out 'product' may have just that effect.  Our own idea that part of what is needed is to at least make the soil -- the research environment ecology -- more likely to engender innovation.  That would mean to down-size, slow down, think more and return research to being more of a profession than an industry.  There are too many professors, too pressured to grind out too many papers or hustle too many grants to keep the administrators and careerists happy.  Too many administrators who, dependent on the cream, need to keep the factory humming.  Time to think, or re-think, or be inspired creatively by odd facts is hard to come by when the pressure is get grants or lose your job.

Better synthetic rather than narrowly technical education is needed, but we haven't been generating the kinds of people who can teach it.  Instead, we have trained a body of academic professors who have been brought up and entrained in, depend on, and hence perhaps can't see or can't afford to see what is actually happening. But despite at least some people pointing the problems out, there is nothing on the horizon that seems yet able to stimulate real change.

All of this is true.  It is also true that the problems are difficult, many if not most faculty have a sincere drive to help public or individual health, high technology is at least somewhat effective and more than just expensive showy toys, and administrators are needed for big, expensive systems.  The methods are often canned in ready-made software, but that doesn't make them wrong, even if we clearly are thinking wrongly in some way.  Until we're shown better, while we do act like fad-following sheep, we do that because that's what we, collectively, know how to do.  And until our jobs and self-esteem are not constructed around the impatient, short-term factory mentality, one cannot expect us to act very differently. 

We do, after all, need to eat, drink, and try to be merry, for tomorrow we die whether or not we like to accept that.