Oh, God! We found you!

Rock star science
The news from Wednesday's press conference with the rock concert attitude is that the deplorably nicknamed God Particle, actually just the Higgs Boson, may have been found.  We seem unable to cease and desist from mixing science and mysticism, material reality and religion.  Despite that very bourgeois habit, physicists are now claiming to have found the elusive ultimate (as far as we know so far) particle.

As reported in The New York Times, physicists say that "the likelihood that their signal was a result of a chance fluctuation was less than one chance in 3.5 million, so-called “five sigma,” which is the gold standard in physics for a discovery." The reason they could say that to "5 sigma" they have evidence consistent with there being a Higgs has to do with the statistics of massively repeatable observations.  They are referring to standard deviations of variable outcomes that have a predicted distribution.  Theoretically, the probability of observing as many or more predicted scatter-tracks (resulting from proton collisions) just by chance, if there were no Higgs Boson at all, can be computed.  Another way to think about this is that it is the level of surprise even biased scientists hyping for their project would experience at seeing what they saw if their theory were in fact wrong.

So, they observed the resulting tracts from zillions of protons bashing against each other, a Large Hadron Dodgem Car operation.  The observed number of these predicted bumps that they saw could happen by chance alone (that is, if there really are no Bosons) once in 4 or 5 million experiments such as the one they did.  This assumes their theory and calculations and measurements are accurate.

If so, the idea is something like this:  A truly fair coin comes up Heads 50% of the time.  But suppose in fact coins are slightly unfair, such that for example the probability of a Heads is not 0.50 but 0.5000001.  To get statistically convincing evidence for that would require something like observing  every flip of a coin in the world for a year.  With that size sample you might  get enough excess Heads to declare that being fair (having no bias, analogous to physics being fairly free of Bosons) was extremely unlikely, and hence that the coin was unfair.  Here you could not say why it was unfair but just that Heads was more likely.  In the case of Bosons, what is being said is that there being no Bosons is convincingly unlikely.  That in itself doesn't prove that the results proved the nature of Bosons, but in this case the physicists apparently predicted how the Boson tracks would differ from chance, not just that they would.

Here, in this kind of evidence, the Boson effects are very small, requiring huge samples to detect something like what was predicted in a statistically convincing way.  Whether something that is statistically significant in this way is also important is another matter (and we've blogged about probability in the past, if you're interested).  In the case of physics theory, the idea is that Higgs is fundamentally important, so it's the smallness of the detectable effect that makes this statistical, not the smallness of the effect in relation to the nature of matter.

Other elusive prey
The Higgs Boson is not the only now-you-see-it-now-you-don't scientific prey lurking in the forest of Nature these days.  In genetics, evidence for effects of traits caused by large numbers of genes is similarly elusive: The search is for genomic sites in which one variant shows up more often in cases (people with some disease or other trait) compared to how often it occurs in controls (people unaffected by the trait)--like Heads coming up more often than Tails.   To detect these differences requires large samples for two reasons.  First, the effects being sought are as weak as the bump of a Boson, and weak effects as we've seen require many repeated observations to detect convincingly.

Second, in the case of genetics, in addition to looking many times at the same site in the genome, we search gazillions of different sites along the genome.  Each site involves an independent test for association with the trait.  If the levels of surprise that you are willing to accept are too weak, you'll be inundated with results most of which will be statistical flukes of your sample--rather than reflecting true causal effects at the tested site in the genome.  This is called the 'multiple testing' problem, and the usual response is to require results to be statistically very significant before you accept them.  The individual prey are so anemic that the state of the not-so-art is to require hundreds of thousands of cases and controls to be an adequate-size study.  It's a confession that the effect will be tiny, but based on the rationale that it will tell us about network interactions in which the cause is involved -- this rationale is largely forced, as we know from experience.

Unlike the Higgs, however, nobody says the weak genetic cause at a given site in the genome is fundamental to the trait, much less to everything in the organism.  And unlike Higgs, each of the many such weak causes is different, whereas the Higgs is the same (everywhere in the universe) so that Hadron is testing the same thing countless times.

In essence, Higgsification is an infinitely repeatable phenomenon, as we're all loaded to the gills with Bosons.  But each individual person is genetically unique, so that genomes aren't really repeatable.

Unified theory
There is, however, one similarity between Hadron physics and human genetics.  No matter what the results, scientists will (1) insist that the same level of funding or even more must still be spent on the Collider and its scientific projects, and (2) that their  prefered theory is still right.  In the latter case, if you are an adherent to the Standard theory, Higgs confirms your view in (statistical) spades.  But even if you are wound up in the string theory, Higgs will be fitted into one of its too-many-to-test dimensions, so you can keep the grants and papers flowing, and physics departments won't be told to unload some of their now-obsolete theoreticians.

Things get subtle in other ways.  Our species wouldn't live long enough for the Hadron folks to run millions of experiments in order that one experiment--but which, nobody would know!-- would, Higgslety-Pigglety yield a result that looked like evidence for the mote in God's eye but was just a fluke.  Equally, some of those countless runs would show no statistically significant evidence for Old Boson even though he was actually there.

In life we see something rather similar.  You are definitely here (MT's hit-counter tells us so!), yet had we stepped back even a few millennia in time, and asked what the chance you'd be here was, it would have been smaller than a Higg's Boson's footprint.  That is, very, very, unimaginably rare events do happen all the time, and they're as real as your life.

This is why statistical evidence is so problematic and yet seemingly fundamental to the way we do science.  If we're understanding the gleeful news releases correctly, the evidence points towards something Higgs-like.  We could just be victims of bad luck (indeed, the second experiment, apparently, gave stronger evidence than the first so that pooling the two to increase sample sizes actually weakened the  overall evidence--though it remains strong).  Right now, it appears more likely that Higgs is there, than not (where God is all this time, is a question for the ages that the state of the world shows that even the folks at Hadron can't answer).

It will be indeed interesting and exciting if this finding is being correctly interpreted and is as is being proclaimed, the most important discovery in a century (after sliced bread, as they say), and leads to undreamt of advances.  Maybe we'll be able to travel on Bosons into the future, to see how long we keep clinging to the current theories of genomic causation.