Wednesday, July 10, 2013

Let's put this subject to bed...

Our Monday post raised quite a stir.  It was triggered by a Science paper about a large mega-study on the genomics of educational achievement, and a subsequent blog post by geneticist Dan Graur.  An issue that we suggested would come up was what we felt was an inevitable interest in using this GWAS result specifically to look for group differences (i.e., especially including 'race' differences) in intelligence.  Even such super-minimal results as reported in the Science paper--trivial genetic contributions to the chosen outcome, educational attainment--can be easily used to justify such purposes.

Our 'policy' on vitriol
Sure enough, we got a few comments expounding upon exactly that point of view, some rather virulently.  You won't see these here, because, for one thing, we didn't want to let that thread get started.  And we don't publish comments that attack individuals in ad hominem ways, nor take blatantly racist stances.  While we don't block disagreement, we do exercise what does amount to a form of censorship.  Those views have plenty of other places to be aired, but not here. 

It is fair enough to disagree, and the blogosphere is a place for opinions to be aired and discussed.  There are few other venues with so wide and fast a reach, a wonderful thing.  Indeed, we use much of our post-space to critique areas in which we feel evolutionary concepts, or genetics and related topics could be differently considered.  We try to do it in the context of the science itself, pointing out issues as we see them.

Of course each study is done by people, and we criticize the 'system' for the various vested interests that drive it.  But we rarely if ever knowingly aim this directly at individual authors in a personalized sense, even if we can and will say when we think someone should know better than to say what he or she said.  We're all human and fallible, however, so it is the issues that are important, not the individuals in our context.

Racism and 'IQ' (here used to represent the panoply of intellectual achievement measures and concepts) have a nasty history.  Polarization is deep and the issues--for those who want to take science seriously--are complex, and the science can and will be used by both those who believe that inequality is justified, and those who believe it's not.

Taking the science seriously
What about intelligence differences, for example?  Are they real, or are they only social constructs by right-wingers?   From a genetic and evolutionary point of view, if you do an IQ study comparing any two individuals, or two groups (say, the left and right sides of a class, two random individuals, two populations you choose to sample--even identical twins) and you find no difference between them, then there is something wrong with your study!  Because of existing variation and new mutational variation, every pair of individuals, and hence every set of indigenous, geographically separate populations that you sample, will differ genetically, and differences will be found across the genome.  Even identical twins are not genomically identical, because every cell division during each twin's life involves the occurrence of some new mutation.

At the moment, we're not considering the manifestly important 'environmental' effects.  But a reductio ad absurdem is that if you don't go to school or can't read you can't score high on a school IQ test.  So, because evolution is the process that got us where we are and is a population process of sorting through variation, you cannot expect exact identity between any two people.  So a study that finds nothing is not done right, uses inaccurate methods of testing for differences, etc.

GWAS knowingly bury genome sites whose differences between cases and controls, for example, are not statistically 'significant', reporting only what passes a specified statistical test.  This is essentially what must be done if one takes a sampling and statistical approach to the subject, as is current standard practice.  We think it's not good science, but that's beside the point.  The point is that when only a few small effects are reported, this does not mean there are no other genomic effects even in that study's data.  Indeed, the idea that most of the heritability--the aggregate genetic effects we find by using trait comparisons in relatives, for example--is 'hidden', really refers to the problem of statistical testing and the intentional ignoring of effects that must be there but don't individually pass a statistical significance test.

To do the science right, or at least in a better way, is a difficult challenge.  Serious issues that may not be compatible with a steady stream of hypothesis-free mega Big Data projects, studies too large and too costly to terminate so the funds can go more productively elsewhere, should be but too often aren't addressed.  But the issues are known, and they are subtle.  These facts are not secret, known only to science critics.  Everyone who cares to think about them, can know them.

An example we mentioned in a response to a comment on Monday's post is this.  In its effort not to be flooded with false positive test results, GWAS buries small, non-statistically significant effects.  The larger the study (assuming properly informed design) the smaller will the undetected effects be, even if they are there in aggregate and indeed are the bulk of the causal variation.  But GWAS and similar findings are widely presented as being far more definitive than they usually are.  For example, not only do genomewide association types of study bury, for practical reasons, the bulk (usually the vast bulk) of genomic effects, but typically many or most or even all clear, previously documented effects are not found in even huge GWAS reports.  How can that be, if the study is so huge?  For example, there are, as we noted on Monday, tens or even hundreds of rather clear-cut genes that when mutated in some ways cause serious IQ impairment.  Yet virtually none of them were found in the large educational achievement study.  Were previous studies wrong?  Are these genes not involved after all?

One answer is that causal mutations simply were not present in those genes in the  specific study sample.  They do exist, at low frequency, in the population, but not in the sampled 'normal' part of the population going to regular schools. The point is that even such huge studies do not represent genomic effects on a trait, nor even those in the population, in a very clear way.  Other samples, in other populations, will (as we know and should expect from what we understand about evolution) typically find other 'hits'.  And this without considering environment.  This is poor epistemology for understanding traits, and it's poor science; or at least, we should be thinking hard about better conceptual ways to understand what genomes do.

Why it's so hard to put this subject to bed!
Well, what about environment?  Reflecting the obvious, if not even perhaps rather ridiculous, state of things, the BBC posted a story yesterday on the major effects on education achievement of going to bed late.  This study was a mere 11,000 strong (compare to the genome mapping study's mega design), yet it easily found substantial achievement effects--far bigger in that sense than the Big Data study.  And it was also clear that they were partly reflective of socieoeconomic status.

Now since most genes--80%, according to gene expression results from the Allen Brain Atlas--are known to be expressed in the brain at some point, they all become potential candidates, and many or most are affected by various environmental conditions (including bedtime, or breast feeding as we described on Monday?).  This means that in any given study only some few genes have statistically detectable effect, and that means the study only reflects its particular sub-sample of the population.  It misses the effects, genuine and present, of countless other gene regions.  At best, it means that such genes did not vary in relevant ways in one's sample, but that does not mean the gene isn't contributing to the trait, just not to its variation in the sample. 

One of the legitimate problems, as well as keep-funding-me rationales for Big Data studies, is that after collecting the gobs of proposed data, later one inevitably learns of things that weren't measured or flaws in the measurement methods.  The investigators then say they 'must' go back and re-contact, re-interview, or re-test all the subjects to add this bit of new vital information.  It is not entirely unrelated that  this is a justification for further funding, and this suggestion is reflected in the fact that rarely (if ever) does the investigator say they need no further work or funds.

So, if breast feeding and bedtimes weren't measured in the current study we've been discussing, and yet they've been demonstrated by other studies to have effects on educational attainment, the results are almost literally worthless.  One might assume that breast feeding could be interacting with the few minor hits that were found, and taking that into account might un-hit those genome sites (and just as likely up the test score for others).  The point is easy to see and is a very serious one.

Beyond the principles, of course you can't practicably recontact everyone in a mega-study (many original subjects will have died and new potential subjects born since the first data acquisition--so the routine ploy is to say we 'need' to study the next generation, etc.).  And environments we live in are very rapidly changing, in literally unpredictable ways and amounts, as lifestyle and medication (and education) fads and fashions come and go faster than a video game.  It's a moving target and this is one reason we think that such massive can't-be-terminated database or survey studies are often not a good way to spend huge amounts of public funds.  They entrench diminishing returns.  There are many such studies that are decades old, well past their proper sell-by dates.

The issues as well as the very ways they are studied, are not being given their due consideration, and in this case interpretation, even by scientists who don't think about these issues very carefully in the rush to do more and larger studies and the like.  We are not working with well-posed questions.  When that is the case, then our epistemology--theory, study designs, and inference--is ripe for questioning.

Science as politics
The key issues here are the epistemic ones of how to identify or even just to define causation, how causation works, and what causal effects are important.  These are fundamental questions in science generally, not just with respect to the study of intelligence.  These aspects of science are inherently subjective.  We have to make judgments about them, no matter how causation really works.  When we make judgments, and the work that leads to them, or follow-up work, or implications of the work involve public policy and the like, then the science is necessarily political.

To accuse a scientist of being just political because of his/her views on a subject--like looking for genomic effects on IQ--is to misunderstand the very nature of the enterprise.  To act as if history provides no guide to understanding the nature or use of science is naive, if not downright societally dangerous.  But often, the right and left wings of our political spectrum accuse science they don't like of being 'just political' and thus dismiss it. Think of 'evolution' or 'climate change'--or IQ.

The truth is that today much of science today is political in this unavoidable sense.  We all pay for it, and what is studied and what is done with the results affect us.  It is, and it should be political.  This is not to say (as some science-studies critiques seem to) that the real world is all imagined and doesn't exist and is a plot by the intellectual elite.  The point is that the real world exists, but how we respond to it, or what we choose to study, as a society are affected by considerations other than scientific study design or analytical methods and other related decisions and techniques.  And we also affect what we learn by how we choose to study it.  Objectivity is our stated goal but is often quite elusive and subtle to achieve.  But nobody seems to want to think about this seriously.  Leave us alone, trust us: we'll do the science right.

The societal decisions we make and what we fund are not just how to respond to curiosity about the world, but to decide which subjects are important enough to us to invest in, or how much to invest.  These are not easy, and we freely express our views here on this blog.  They are subjects that should be taken seriously. It is perfectly legitimate to say that this or that kind of investigation are not in society's interests to support, or even to allow (we don't allow torturing of prisoners for research purposes, for example, and institutional review boards are charged with making decisions about whether other less obviously egregious but potentially problematic studies should be allowed).

So we certainly do color our own MT blog posts with considerations that we think are relevant in this respect.  These considerations, such as where funds should be spent, are openly, not deceptively, political.  Obviously, vested interests, cultural practices, professional needs, careerism, desire to improve society, and genuine quest for knowledge are all at work.  Rather than energetic but reflexive reactions, much less vitriolic responses, what we try to urge is that we take nature more seriously in our attempt to understand how she works.  That's a lot harder than just business as usual.

No comments: