Monday, January 26, 2015

What's 'precise' about 'precision' medicine (besides desperate spin)?

Not very long ago we were promised 'personalized genomic medicine'. Surely you remember.   It was a slogan like any advertising slogan and if you read the fine print (the caveats, if you can find them) you'll see various safety valves.  Medicine has always been 'personalized' but the implication was that we'd be treating everyone in ways that are specifically dictated by what we find in their genome (whole DNA sequence of their 'constitutive' or inherited genome).

The idea was an advertising or promotional way of lobbying for funds for the belief system that genomes cause everything (except, perhaps, the final Super Bowl score...but even that, well, if the quarterbacks' and receivers' skills are--as surely they must be--genetically determined, maybe we can even predict that!).  Of course, when someone carries a particular genotype at some locus with a strong effect, and many of those are known, a clinician should, indeed must, take that into account.

But that is nothing new, and has been the business of the profession of genetic counselors and so on (not necessarily of online DNA businesses one might name).  That sort of personalized genomic medicine is no more novel than 'evidence-based' medicine, which is another slogan, this time perhaps for the for-profit HMO businesses who want to dictate how the doctors in their stable they control treat their patients, and it's nominal objective is to eliminate poor doctoring, which is good, but anyone not thinking it's basically about profits is willfully naive.

Anyway, now we're seeing a new slogan, so mustn't that mean we have successfully achieved the goal of 'personalized genomic medicine'?  Obviously, if we have any accountability left in government spending, even under Republicans, if we didn't solve the previous objective, what's the justification for a new one, much less expecting Congress to allocate the funding for it?  A cynic (but not us!) might say that the lobbying aspect of biomedical research is now onto the next slogan, changing the packaging even if the product's not really changed.  One has to keep changing slogans if one wants to keep customers' (and Congress') attention so they'll keep giving you money.

The new slogan is 'precision medicine'.  Sara Reardon says this about that in Nature:
The agency seems to have been planning the effort for some time, listing 'precision medicine' as one of its four priorities in its 2015 budget proposal; another was 'big data'. Other government agencies are also expected to participate, as may some private companies. There is no word on how much the initiative will cost, but details are likely to trickle out as Obama prepares his budget request for fiscal year 2016, which is due to be released on 2 February.
Wow!  This will replace, one has to assume, all that 'sloppy' medicine of the past, just as 'evidence-based' medicine presumably replaced 'lack-of-evidence-based medicine'.  Now, docs will know precisely what ails you and precisely what to do about it, and this will precisely involve genetic approaches since that's all NIH's leadership seems to understand.

OK, OK, so we're (again) being snide.  But not just snide.  You ask yourself what one might mean by 'precision' medicine?  Does it mean 100% accurate, no mistakes?  Of course not, so then, what?  If it means the doc does his/her best, is that anything new or something to write home about?  Surely Dr Collins isn't suggesting that he's been imprecise with his promises about genomics.  Surely not!  Instead, the claim is that we'll be able to look at your genome and hence know precisely what is in your future and what to do for (or to) you as a patient.   Anyone who knows anything about genetics knows that, with some clear-cut but generally rare exceptions, that's bollocks!

What does 'precision' mean (if anything)?
The goal of precision genomic medicine sounds laudable, though if it refers to everything being precisely based on genes, that would be the only thing that's new.  In some venues, at least, the idea has instantly received the ridicule it precisely deserves from the get-go. At least, conscientious doctors have always done precisely as well as they could, given their knowledge at the time.  So, what must be meant is that now genetics will let us treat patients in a way that is, finally, really precise!

But wait: what does the word 'precise' actually mean? Well, look it up. It means 'marked by exactness or accuracy'.  OK again, that sounds great.....or does it?  Does even Francis Collins, your genome's best friend, really believe that we'll have exact diagnosis or treatment based on genome sequences? Or what about 'accurate'?  Which doesn't necessarily mean correct, or appropriate -- just targeted at a given spot.

'Precision' can refer to perfection or exactness.  Or it can refer to something less, some kind of within-knowable error.  Proper science always deals in precision, but properly, by specifying the degree of accuracy.  One says an estimate is precise to within x percent, based on the current data.  But a risk of, say, 5% can be precise to within , say, 1 or 1/10 of a percent or more or less.  It's not all that reassuring that your genome gives your doc something like that unless the range is narrow and reliable.  In general that is the kind of meaning we can assign to 'precision', when done honestly and honorably, but it's far from the scientific reality in this field.

Indeed, when the precision of estimates is presented properly, one is in essence expressing inexactness--of a machine or instrument or estimate.  To claim that we're going to be using NIH research funds to generate precision medicine is saying that we're going to do the best we can.  One can only hope so!

Precision and accuracy are adjectives that do not in themselves mean anything until the degree is stated, along with how that degree is known and by what criterion.  Does it relate to replication of a process and similarity of results?  Does it mean what will happen or what might happen with some specifiable accuracy or probability?  If you don't state that, you're just playing empty word games.

Worse than that, honest research and clinical treatment has always been 'the best we can' at any given time.  Likewise, genome-based prediction, usually very imprecise both in the sense of being inaccurate and also of being of very poorly known degree of inaccuracy, is nothing new.  It's nobody's fault, because the genome and its interactions with the environment are complex.  One hopes that whatever real genomic information on risk, response to treatment, diagnosis etc. we have will be as precise as possible, in the proper sense of the term, and that new knowledge from research will increase that precision.

But it is dishonorable to imply that this is something new and different, and to suggest even implicitly that genome sequencing and the like are leading us to anything close to what most people think of when they hear the word 'precision', is anything new or is generally even in the cards.  Especially following on the previously largely vacuous promise of 'personalized' genomic medicine.

It is very misleading to suggest otherwise.  It takes guts and ruthless lobbying.  'Precision' is literally almost meaningless in this context!

The million genomes project
In the same breath, we're hearing that we'll be funding a million genomes project.  The implication is that if we have a million whole genome sequences, we will have 'precision medicine' (personalized, too!).  But is that a serious claim or is it a laugh?

A million is a large number, but if most variation in gene-based risk is due, as mountains of evidence shows, to countless very rare variants, many of them essentially new, and hordes of them perhaps per person, then even a million genome sequences will not be nearly enough to yield much of what is being promised by the term 'precision'!  We'd need to sequence everybody (I'm sure Dr Collins has that in mind as the next Major Slogan, and I know other countries are talking that way).

Don't be naive enough to take this for something other than what it really is:  (1) a ploy to secure continued funding perpetrated on his Genome Dream, but in the absence of new ideas and the presence of promises any preacher would be proud of, and results that so far clearly belie it; and (2) a way to protect influential NIH clients with major projects that no longer really merit continued protection, but which will be included in this one (3) to guarantee congressional support from our representatives who really don't know enough to see through it or who simply believe or just want cover for the idea that these sorts of thing (add Defense contracting and NASA mega-projects as other instances) are simply good for local business and sound good to campaign on.

Yes, Francis Collins is born-again with perhaps a simplistic one-cause worldview to go with that.  He certainly knows what he's doing when it comes to marketing based on genetic promises of salvation.  This idea is going to be very good for a whole entrenched segment of the research business, because he's clever enough to say that it will not just be one 'project' but is apparently going to have genome sequencing done on an olio of existing projects.  Rationales for this sort of 'project' are that long-standing, or perhaps long-limping, projects will be salvaged because they can 'inexpensively' be added to this new effort.  That's justified because then we don't have to collect all that valuable data over again.

But if you think about what we already know about genome sequences and their evolution, and about what's been found with cruder data, from those very projects to be incorporated among others, a million genome sequences will not generate anything like what we usually understand the generic term 'precision' to mean.  Cruder data?  Yes, for example, the kinds of data we have on many of these ongoing studies, based on inheritance, on epidemiological risk assessment, or on other huge genomewide mapping has consistently shown that there is scant new serious information to be found by simply sequencing between mapping-marker sites.  The argument that the significance level will raise when we test the actual site doesn't mean the signal will be strong enough to change the general picture.  That picture is that there simply are not major risk factors except, certainly, some rare strong ones hiding in the sequence leaf-litter of rare or functionless variants.

Of course, there will be exceptions, and they'll be trumpeted to the news media from the mountain top.  But they are exceptions, and finding them is not the same as a proper cost-benefit assessment of research priorities. If we have paid for so many mega-GWAS studies to learn something about genomic causation, then we should heed the lessons we ourselves have learned.

Secondly, the data collected or measures taken decades ago in these huge long-term studies are often no longer state of the art, and many people followed for decades are now pushing up daisies, and can't be followed up.

Thirdly, is the fact that the epidemiological (e.g., lifestyle, environment...) data have clearly been shown largely to yield findings that get reversed by the next study down the pike.   That's the daily news that the latest study has now shown that all previous studies had it wrong:  factor X isn't a risk factor after all.  Again, major single-factor causation is elusive already, so just pouring funds on detailed sequencing will mainly be finding reasons for existing programs to buy more gear to milk cows that are already drying up.

Fourth, many if not even most of the major traits whose importance has justified mega-epidemiological longterm follow up studies, have failed to find consistent risk factors to begin with. But for many of the traits, the risk (incidence) has risen faster than the typical response to artificial selection.  In that case, if genomic causation were tractably simple, such strong 'selection' should reflect those few genes whose variants respond to the changed environmental circumstances.  But these are the same traits (obesity, stature, diabetes, autism,.....) for which mapping shows that single, simple genetic causation does not obtain (and, again, that assumes that the environmental risk factors purportedly responsible are even identified, and the yes-no results just mentioned above shows otherwise).

Worse than this, what about the microbiome or the epigenome, that are supposedly so important? Genome sequencing, a convenient way to carry on just as before, simply cannot generally turn miracles in those areas, because they require other kinds of data (and, not available from current sequencing samples nor, of course, from deceased subjects even if we had stored their blood samples).

And these data will be almost completely blind to another potentially very important genetic causal process, that of somatic mutation.  Tomorrow we'll discuss that issue.

Of course, there are many truly convincing genetic factors that are clearly relevant to know and to use in diagnosis or treatment decisions.  Those are precisely the factors to test for, investigate, intervene with and so on.  NIH should be closed down entirely if they are aiming at anything different (and no one suggests that, in general, they are).  Even if rare, or 'orphan' disorders, they are ones for targeting engineering preventive or curative approaches to; if science is good at anything, it is engineering.

Precisely what will the current proposed work yield, then?  It will yield an even better picture of how wasteful this sort of perpetuate-every-big-study-you-can-identify project will have been.  That is, at least, one precise  prediction!

Caveat emptor!  The Ayn Rand factor: don't mistake it for science
Everybody naturally wants precision-based medicine, using genes in those areas in which that is wholly appropriate.  But what we should expect is that NIH puts its resources precisely where they will do the most good for the investment.  Those, including Congress, who think that NIH has been doing that in the recent past are precisely those who should start paying closer attention.

If you are among those who have paid attention to the miracles NIH officials, Dr Collins in particular, have been promising and the language they have used, now for over 20 years, then you should suggest that they leave NIH and get new jobs in a place where truthfulness is not part of the deal: Madison Avenue. Unless, of course, simply finding a rationale for keeping the funding tap open to existing clients is the real underlying objective.

Some readers may say enough winging about this is enough!  After all, genes are important so this will be good science even if promises are exaggerated.  There will be good science, certainly, but this HyperProject will undermine the idea of nimble science, driven by ideas rather than empires.

Science does involve money and so is never too far from underlying politics. In a largely Republican environment, dancing to the ghost of Ayn Rand, it will be interesting to see if those now in power keep indulging the 'haves' in science.  One might expect them to, but at the same time, this is, when you look closely, largely a welfare project for the science in-groups, to keep life in large, long-standing, tired projects well past their point of diminished returns: like bailing out rusting industries, the long-term projects once found useful things but are now clearly degraded from good cost-benefit  profiles.  So one can ask if even the Republicans are paying attention?

A Big Data bailout will, of course, preserve jobs and career status for the main recipients.  The Old Boy networks in science were to some extent dismantled by the democratization of funding that began in NIH and NSF around the 1980s.  Peer review included peers of both genders, racially distributed and funding geographically dispersed.  It was never perfectly equitable, but it was much more open and democratic than the back-room sense that seemed to have prevailed before.

But the new Old Boys (and now Girls, too, just as acquisitive as the Boys) are back with a vengeance!  As always, those at the top, with the proposed bail-out project extension, will hire many minions of workers, including technicians, post-docs and other well-trained scientists.  Those are jobs, certainly, but as before they're largely located in the elite universities that have long had a big grasp on funds (and have munificent private sources they could use instead of feeding so hungrily at the public trough).  You can voice your own view about whether this elitism is what's best for science or not.  But no matter, welfare for scientists and technicians, controlled by the Big Lab aristocracy, is largely what's afoot--again.

Aristocracies maintain themselves by making enough people dependent on them--the research cogs.
If you have a liberal bent of mind, as we tend to, you can't object to the use of public sector resources so people have jobs, though inside tracks for science-related people isn't exactly democratic.  But what the welfare-for-tired-projects will do, as adverted to above, is to deprive even these clever inside people of chances to be innovative, and for science to be more nimble.  That's because the army of scientists and staff, those on the hundreds-of-author papers, are forced by this system to be cogs in the Mega-wheel of Big Data projects.  That's not good for science.

For, not against
Our arguments are not against science, but for it.  They are for nimble, fleet-footed science with a fair, idea-driven marketplace rather than institutionally inertial one.  And, also as noted earlier, a million sounds good but isn't nearly enough for some of the implied success and seems more likely to be intentionally setting the table for the future of this welfare system--whole population sequencing! Why not?  It would be rather surprising if the Director, given his track record, hasn't got this in his back pocket for when the current catch-phrase is worn.

An irony is that our comments here might be interpreted as dismissing the causal importance of genetics in the nature of organisms and their evolution.  In a sense, our message is the opposite: it's that by building genetics into the sociopolitical institutional structure of science, and hence its particular welfare or self-maintenance system, we routinize what isn't yet well enough understood to be routinized.  We trivialize genetics in that way, the opposite of what should be done.  We benumb minds that should be sharpened by facing an open, rather than channeled frontier.

One thing, though: this is not a shell game!  It's all being done in plain sight.  You and everyone who thinks about it, knows what this is.  We personally are newly retired and have no dog in the fight. But one would expect that sooner or later a wide community of scientists will tire of Dr Collins' continual feeding of his narrow ideology (or his dependents, view it how you wish), for lack of better scientific ideas.  If the victims whose careers and ideas are not being protected by this welfare system don't care enough, don't act, or can't find a way to resist by credible challenges to the status quo, the status quo will remain.  It's that simple.


David J. Littleboy said...

Hmm. I think you've gone a bit overboard. I agree that "precision" is a badly chosen word, but I'm hoping that genomics will lead to the following sorts of improvements. A given medication will produce improvements in such and such a percentage of patients (i.e. have a specific number to treat) and side effects in a different percentage of patients (number to harm). If one could use genomic information to increase and decrease those numbers even just a bit, it'd be big. (This could be enormous in cancer treatment.) As a white male of a certain age and medical history, it turns out I have a 12% chance of a heart attack over the next 10 years (according to the Framingham calculator). If I am going to have said heart attack, it'd be real nice if I'd been taking an aspirin every day. But aspirin has a certain chance of causing a variety of other problems. If genomic info could tell me even slightly more accurately how likely those other problems are, I'd have a better way of deciding what to do.

In real life, though, my bet is that genomics is going to be a stupendous fizzle for the simple reason that all the low hanging fruit is obvious from family history, and the higher fruit is really tiny (so tiny, it can't be seen in family histories, doh). Sigh.

Even worse, the kinds of studies required to figure out genomic effects in number to treat or number to harm values are expensive, time consuming, and unlikely to get done. Sigh, again.

Michael Finfer, MD said...

Precision is a badly chosen word. It refers to the reproducibility of results, that if you do a test more than once, you will get, more or less, the same result.

I have been wondering where all this genomic stuff is going. If any of it pans out, all it will allow us to do is stratify patients by risk. I doubt that it will allow us to predict with any accuracy whether a five year old will eventually have an myocardial infarction, much less a premature one. I, of course, am not referring to those rare patients with single gene abnormalities, but to the rest of the population.

However, I have always had hope for the application of sequencing of simple organisms in reducing the amount of time that it takes to identify the cause of an infection and to determine which antibiotics it is susceptible to. That can currently take anywhere from a day or two to two or three months for an organism like Mycobacterium tuberculosis. Even for that, we are nowhere near ready to go. These techniques are so sensitive that it might be difficult to sort out what is going on, especially in a typically non-sterile specimen like sputum, where you will get many (hundreds or more?) of organisms identified and then have to sort out what you have to treat.

Sometimes I wonder if any of the most prominent researchers remember anything about the art of diagnosis. Some of them seem to think that everything can be computerized, and I don't see that happening in my lifetime.

Ken Weiss said...

Reply to Drs Littleboy and Finfer:
Overboard? It's bluntly worded, not smoothly written or elegant perhaps, but it's nothing compared to the full-court press being applied by Francis for this.

This is a zero-sum 'game', too. In tight times one might even say negative-sum. What is funded for one purpose, directly deprives other aims of funding. In tight times, with many careers (and hoped-for careers) in the balance, it's not time to mince words.

Also, I think the statements we made are, basically, cogent and -- yes, NIH! -- evidence-based!

Science can't predict discoveries because Nature challenges us in deep ways. But we can hope to raise the odds of major findings by exploring new paths.

Playing it safe and the inertial, almost anti-entrepreneurial tactics that we're seeing are totally understandable tactics but which don't seem like the best way to get science for the public good.

Michael Finfer, MD said...

I did not mean to imply that there is no promise and that no research should be done. There is promise, but there is also hype, and I see no way that genomics can live up to the hype in the foreseeable future.