Last week we all heard about the new recommendations concerning predicting and preventing heart disease. An online calculator was going to be used to predict risk, based on past experience of a study cohort, and if we were found to be at risk, we'd be advised to start taking statins. Based on various risk factors -- BMI, whether we smoke, etc., -- the calculator would advise our doctor if our risk of heart disease was greater than 7.5%, and if so, that would trigger the writing of the prescription. Estimates were that this was going to lead to 1/3 or more of American adults taking statins for the rest of their life.
We had various reasons to question this recommendation, as we wrote then, and that was even before new issues have come home to roost. As reported in the NY Times yesterday (and in The Lancet today), experts are now showing that the calculator is way over-estimating risk, which would mean millions more people on statins than the generous new guidelines themselves would recommend. Some leading cardiologists are calling for a halt to implementation of the new guidelines until the calculator issues get sorted.
But apparently this shouldn't have come as a surprise to the cardiac community -- two Harvard cardiologists, authors of the Lancet paper, warned a year ago that the calculator overestimates risk. The problem seems to stem from the use of old data to do the estimations. Ten or more years ago more people smoked, and developed cardiovascular disease earlier than they do now. Risk estimates on that background now overestimate the effect of factors such as blood pressure and cholesterol because additional background factors, also a component of risk, have changed. That is, there are confounding variables that affect risk, whose frequency in the population have change, but the calculator doesn't take this into account.
Or at least that's probably the problem. It may be even more fundamental than that. Decades ago, a very well-known cardiovascular disease epidemiologist, Reuel Stallones, used to say that heart disease rates had both risen and fallen in the 20th century for reasons that were not understood. Diet, exercise, smoking, cholesterol, none of the obvious risk factors explained either the rise or the fall, and it's still true.
However, the risk calculator problem raises another truly fundamental issue that pertains to this sort of risk prediction in general, not just to heart disease. We have several times noted that one essential flaw in the whole concept of risk estimation based on the kinds of studies that are done, is that risk is estimated retrospectively, from a study sample's past experiences, but what we want are prospective risks: yours and mine for the future, not the past. But future experiences, mainly here involving lifestyle environmental factors, are inherently unpredictable (In case you missed it, that's inherently unpredictable).
The critique of the new recommendations, in an unusual way, showed just this problem. Risks were estimated for the new calculator from past data, but used to estimate risks subsequently. However, since lifestyle risk has changed (less smoking, for example) our ultimate experiences can't be adequately predicted from risks based on the earlier experiences of the cohort used to estimate risk. Whatever would lead properly cognizant epidemiologists to think that things would be different for the real future?
Indeed, this shows the stubbornness of our clinging to kinds of statistical association mechanisms, the belief in big-science, the haste, and so on that plagues much of what is afoot these days. To a great extent, we do what we know how to do, the problems are very challenging and often risks are small in absolute value so that, with our approach, we do look to very large studies. But we also stick with this rather than slowing down, taking a deep breath, and really re-evaluating what we face. This is the issue we often write about: the need for a deep change in our scientific concepts or methodology, not just keeping the research factory humming.
Plus, we already know a better way to prevent heart disease, and that is lifestyle choices, exercise, not smoking, eating more vegetables than fats and meats. Statins are largely generic drugs now, but they are still an expensive way to prevent illness -- people without disease become patients (see Jim Wood's August post on how he became sick), they are in the medical system, requiring not only drugs for life but testing and follow-up testing and so on. And, statins aren't benign drugs; they do have side effects, including muscle pain, liver damage, diabetes. The industry's credibility is at stake now with this calculator issue, but there are other reasons to question their word about going on statins.