Science is interested in understanding the world as it really is....but, surprisingly, only up to a point! You have to play by the inferential rules, like how to design experiments, evaluate data, and so on. But you also usually have to stay within bounds. That means, you have to stay within the limits of what is considered legitimacy at any given time. And if you don't, you'll get a critique like this one,
“It’s craziness, pure craziness. I can’t believe a major journal is allowing this work in. I think it’s just an embarrassment for the entire field.”
We're not talking about cheating here. We're talking culture, and that means within the tribally accepted belief system. Currently, the science tribe doesn't accept supernatural explanations, so we don't accept hypotheses based on divine intervention in experiments. Thus most scientists say that religion is outside the realm of science, and at present so is consciousness. Subjective experience is also not viewed as objective science!
Of course, what is 'supernatural' is what we don't know how to understand. But if we discover some new factor or force--as happened with the use of telescopes and microscopes, or electromagnetism, the new things are melded in with the known, and hence natural world. No longer supernatural.
At any given time, people tend to believe that all fundamental forces of that sort are known, even if we don't know the details of how they work in various cases (such as predicting traits from genes? future climate? electron orbits?). But if current theory is correct, isn't the world made entirely by molecules and energy and don't they follow universal rules that intertwine them with each other (like gravity, filling space and time)? If that's the case, then nothing is truly independent of anything else, and things we observe could be causally connected in ways we didn't expect.
Heresy?
A paper soon to be published in a highly respected psychology journal, after standard peer review, that claims that something of this order occurs to explain extra sensory perception (ESP, or psi as it's called these days). For example, physics routinely invokes 'entanglement' by which particles in different places can affect each other in ways such that one particle 'knows' the status of the other. So why can't information be transmitted and hence received and interpreted elsewhere, such as by one person perceiving such things without any standard physical evidence of information transfer?
This paper, described today in the NYTimes, and freely accessible online -- and the subject of discussion for some weeks already in the blogosphere -- is a report of 9 different trials of psi. A thousand Cornell undergrads were asked to perform different tasks -- guessing which screen would show an erotic picture before the computer had randomly selected it, for example -- and the results interpreted by the investigator as showing that 'precognition' exists.
The rationale for this decade of experimentation seems to be that what we know about physics makes it possible
The development in quantum mechanics that has created the most excitement and discussion among physicists, philosophers, and psi researchers is the empirical confirmation of Bell’s theorem [see the paper for citations], which implies that any realist model of physical reality that is compatible with quantum mechanics must be nonlocal: It must allow for the possibility that particles that have once interacted can become entangled so that even when they are later separated by arbitrarily large distances, an observation made on one of the particles will simultaneously affect what will be observed on its entangled partners in ways that are incompatible with any physically permissible causal mechanism (such as a signal transmitted between them).and evolutionary theory makes it plausible.
If psi exists, then it is not unreasonable to suppose that it might have been acquired through evolution by conferring survival and reproductive advantage on the species. For example, the ability to anticipate and thereby to avoid danger confers an obvious evolutionary advantage that would be greatly enhanced by the ability to anticipate danger precognitively.Well, of course, if this were so it would also not be unreasonable to suppose that most species in addition to humans would have this ability, and that a trait with the huge selective advantage that psi would surely have would not be as difficult to detect -- or as intermittently useful and reliable -- as it's proving to be.
And, the paper is getting just the kind of broadside critique any heretical challenge to accepted wisdom will get. The quote above, for example, about these results being just crazy is taken from the story in the Times, is from a Ray Hyman, emeritus professor of psychology at the University of Oregon and described as a 'longtime critic of ESP research'. Of course, maybe he's a reflex, superannuated critic. Because there are also those who are applauding the results -- finally mainstream science shows psi exists. But, as is generally the case, reactions to this paper tend to be based on people's preconceived notions about ESP rather than the science. Reactions as predictable as, well, as psi!
Or just bad science?
A rebuttal of the methodology is already available online, presenting a number of explanations for the apparent positive results of this work, including that it's a confusion of exploratory and confirmatory data (that is, the investigator refines his testing based on previous findings and doesn't correct for this), and that a Bayesian analysis that evaluates the plausibility of new results based on what has been found or assumed before, shows that none of the results are significant after all.
Indeed, the idea that psi even exists, much less what it is, is has been thoroughly controversial, or rather, rather thoroughly dismissed by scientists (in our current tribe). Mostly, it's been for good reason. Claims of the phenomenon have largely been unsupported by adequate evidence. Frauds and trickster showmen have been deeply implicated in ESP shows, seances, and the like. Tarot card readers are clever at making educated guesses and safe predictions. So why should we believe any such claims? Is invoking 'entanglement' to explain psi any different than other after-the-fact true-sounding WAGs (wild ass guesses)? Or is it the right kind of explanation, but simply far ahead of what we actually know about what we call 'entanglement'?
History shows that some such things, like alchemy and phrenology, have presaged what became legitimate science, even if their initial premises were guesses based on poorly observed aspects of Nature. Many accepted ideas, like earth-centered astronomy or the four humours theory of medicine, have clung on for centuries even though clearly wrong (we now know). If someone actually shows that psi is 'real' that will mean showing the nature of the mechanism or that it can be systematically repeated, or something of that kind. Then, it will become part of the normally accepted world. The skeptics will be laughed off as having been conservative Luddites impeding progress.
Or both?
On the other hand, hucksters and legitimate scientists alike have offered countless explanations and theories that went absolutely nowhere. And the claims of studies of psi find only slight statistical supporting evidence (making the big assumption of no kinds of bias or misapplication of significance tests, such as how truly multiply blinded participants and observer were, whether multiple testing was corrected for, and what 'significance' level--p value--should be used). Why should the evidence be so weak? In this context, the author's totally Kiplingesque 'evolutionary' Just-So explanation supports suspicion more than it supports the hypothesis, because it's so egregiously trying too hard, far too hard, for something that involves a participant and a computer (of course some day we may find a million year old computer fossil). If humans evolved for anything, perhaps it was to be gullible!
Nonetheless, pseudo psi-ence aside, there remains the legitimately serious question of how we could ever tell psi from sound, and how can we decide the limits of acceptable claims, of what constitutes scientific understand of the real world? This has been an issue, often unstated, throughout the history of science. In a way, it's what keeps science so conventional and incremental in daily practice, but in the long run what makes science interesting.
23 comments:
http://www.youtube.com/watch?v=fn7-JZq0Yxs
And then there are _those_ kinds of methodological problems. Thank you, Holly!
I'm surprised that people with psi were able to be "blinded" to make the experiment scientific!!!
Also, neurons are a bit larger than quantum. A bit complicated for entangling.
Also, were the erotic pictures (which were the ones psi-people were best at detecting) ever previously in contact, physically, with the psi-people? Because wouldn't they have to be to be in order to be quantumly entangled?
Very weird ;).
Seriously, if esp is real then we need to criticize this paper for not being able to "blind" the participants to the study.
That is an important point about the impossibility of this being a blind study.
Several other important points -- on the first pass, only women passed the erotic picture test. So, Bem decided the pictures weren't erotic enough for men, so he pulled some better ones off the web. Which seemed to have worked -- well, if one considers 53.1% correct to mean anything important, statistically significant or not. But shouldn't he have foreseen this problem with men and erotic pictures? Not to mention that his solution surely complicated his analysis (the exploratory vs confirmatory data problem).
I'd like to learn more about the exploratory vs. confirmatory data problem.
Erotic internet pictures cause computers to emit some sort of frequency (compared to a blank screen) that well-trained consumers are tuned to without realizing it?
Gosh, speculating is so much fun! I'm tempted to become a psi-researcher!
Oh wait, I'm a paleoanthropologist already.
There's at least a start to this in the rebuttal paper, here, starting on page 3: http://dl.dropbox.com/u/1018886/Bem6.pdf
"As such, there is nothing wrong with fishing expeditions. But it is vital to indicate clearly and unambiguously which results are obtained by fishing expeditions and which results are obtained by conventional confirmatory procedures. In particular, when results from fishing expeditions are analyzed and presented as if they had been obtained in a confirmatory fashion, the researcher is hiding the fact that the same data were used twice: first to discover a new hypothesis, and then to test that hypothesis. If the researcher fails to state that the data have been so used, this practice is at odds with the basic ideas that underlie scientific methodology"
In fact, Bem did do a lot of fishing around in his data, including throwing out outliers to see what happened, and regrouping them and playing around with which pictures gave him the best results and so on. Not that that's necessarily wrong, but he needed to have corrected for it, and as far as I can tell, did not.
Holly, do physicists actually understand quantum theory? Why should psi-chologists be expected to understand their elusive subject any better?!
psi-chologists! love it.
They all just need to try harder. Way harder. C'mon! It's so easy.
Not sure if you ever experience this, but it seems when you have to make a guess at something and two answers randomly pop into your mind you almost always say the wrong one while the skipped over answer was actually correct. Then there is the "I was thinking that was the answer!" Maybe we're evolutionarily programmed to hide our psi so we're not killed off for being a witch. Haha! Just a thought. And untestable.
Untestable?
http://www.youtube.com/watch?v=zrzMhU_4m-g
Computers are more erotoresponding because most programmers are men. Based on all of this 'blue' discussion, one wonders if mega-doses of Viagra could amplify the signal or be used to test various peoples' level of psi-ability.
The idea of quanta being a tad smaller than a neuron is right on the mark. How could quantum entanglement have sending pattern enough to be detected as such by a brain: each brain is wired differently, too (and each computer may be different).
Holly likes psi-entists to paleoanthropologists, presumably because they are good storytellers, only sometimes constrained by the facts. But at least paleo-Just-So storytellers have some material to show as part of the entertainment, while Just-Psi storytellers only have very small p-values. Better than the response meters on some of the males that were too small to measure, or one might say came up short.
The interesting thing about this, and the arsenic and Space-Life stories, is that anybody publishes them. That really proves that science and the media are entertainment phenomena, and unfortunately that is spreading to other sciences, certainly including genetics as well.
Brilliant, Holly!
Here's a related 800-page tome I hope review when I somehow have free time: Irreducible Mind: Toward a Psychology for the 21st Century (2006/2009) by Edward F. Kelly, Emily Williams Kelly, Adam Crabtree, Alan Gauld, Bruce Greyson, and Michael Grosso.
Here's the blurb from the book:
"Current mainstream opinion in psychology, neuroscience, and philosophy of mind holds that all aspects of human mind and consciousness are generated by physical processes occurring in brains. Views of this sort have dominated recent scholarly publication. The present volume, however, demonstrates—empirically—that this reductive materialism is not only incomplete but false. The authors systematically marshal evidence for a variety of psychological phenomena that are extremely difficult, and in some cases clearly impossible, to account for in conventional physicalist terms. Topics addressed include phenomena of extreme psychophysical influence, memory, psychological automatisms and secondary personality, near-death experiences and allied phenomena, genius-level creativity, and 'mystical' states of consciousness both spontaneous and drug-induced. The authors further show that these rogue phenomena are more readily accommodated by an alternative 'transmission' or 'filter' theory of mind/brain relations advanced over a century ago by a largely forgotten genius, F. W. H. Myers, and developed further by his friend and colleague William James. This theory, moreover, ratifies the commonsense conception of human beings as causally effective conscious agents, and is fully compatible with leading-edge physics and neuroscience. The book should command the attention of all open-minded persons concerned with the still-unsolved mysteries of the mind."
If after my literature research I would conclude that the researchers compiled compelling evidence for a dualistic theory of the mind, then I would refer to the evidence as a philosophical conjecture or a parascience conjecture. I suppose such research might point to an exotic physics of consciousness that has a root origin apart from chemical reactions in neurons. But unless we could analyze the substance of such an exotic physics, then I wouldn't say that the researchers are restricting themselves to the scientific method, but a philosophical conjecture could be reasonable.
When it comes to published psi research (one branch of psychical research) from major universities such as cited in original blog post, I've seen only slim statistical margins that don't impress me.
Anne's fishing expedition comments are cogent because that is exactly what all the 'Omics' fads are about. It's an avowedly proud denial that we need to have hypotheses. And in legitimate science as well as questionable science, there are problems with fishing expeditions, and they have to do with the informal number of tests that are done.
As to Jim's comments and quote from the book, the problem with things that we cannot explain in normal physical terms does not imply that the phenomena are of other than physical nature: our ignorance of one kind of explanation is not an argument for any other specific hypothesis. That's the mistake of Intelligent Design, too.
That, of course, doesn't mean there aren't physical processes that some day will account for psi phenomena. The first problem is the skepticism of results--if the statistics or study designs are sub-standard or flawed, then (as many in science argue) there isn't any psi phenomenon even to demand explanation!
If there are some such phenomena, then we should either eventually be able to explain them in normal physical terms, or to show that they force new ideas about physical terms, or to show that something we haven't yet considered 'physical' is responsible.
But then the phenomena become normal ones, not supernatural ones. The issue we raise here is that it's hard to know when the data say something real that even has to be explained.
Jim,
I failed to respond to your saying that what this could be called is philosophical or parapsych. conjecture. I don't get your point, I fear, but that is certainly something scientists would be comfortable with....but it doesn't imply anything more than just conjecture. Scientists defending regular science would say that instead of that conjecture they think sooner or later these phenomena...if they actually exist and are not huxterism or mis-used statistical analysis...will turn into ordinary material phenomena.
Ken, I appreciate your feedback. I'm trying to apply an approach of cosmology by John Leslie to parapsychology. I also hope to one day develop the approach into a historical method for contemporary religious experiences. And I hope to avoid falling for scientific research with poor methodology and insignificant test results.
Regardless of how little you could understand of what I wrote in my previous post, you helped me. For example, I suppose if empirical experiments would ever indicate compelling evidence of "non-chemical" consciousness in a dualistic mind, then it would be become part of science. But if only compelling historical evidence indicates a dualistic mind, then it would only be part of philosophy and history. On my part, I need to do a lot more work on this before I can pound out a quality paper, but your blog post perked ideas in the back burner of my mind.
I am a long-time acolyte of Bem: self-perception theory was and is brilliant. But this? Utter nonsense. This quotation: “It’s craziness, pure craziness. I can’t believe a major journal is allowing this work in. I think it’s just an embarrassment for the entire field.” is not over the top or untoward or uncalled for: it is the correct response. We all know that Bem's results are nonsense. Seriously. We can't hide behind walls of seeming objectivity, stroking our chins, and saying "well, that is what the data say", because as we all know the data don't speak at all.
This paper is nonsense, and should not have been published, not because we are close-minded bigots, but because it is pseudo-science, bullshit in less politically-correct terms. It hides behind a wall of stupid ("ignorant" would imply some charity) statistics, ridiculous research designs, and less than clear protocols. I can't see how it made it through the review process, but as it did, the reviewers and the editor should hang their heads in shame. This is sham science, stupid science, and a pox on all that we do. As an experimental psychologist, I am embarrassed, but more deeply ashamed. I will have to spend the next decade digging my discipline out of this fucking hole. Thanks.
Thanks, John. We tried to use this paper to make a point that this paper didn't deserve to be used to make, that it can be difficult to tell whether something's bad because it's bad, or that it just seems bad because we just don't like the idea. We stand by the general point, but agree, and are happy to have you say so in no uncertain terms, that this paper is just bad science.
I guess the problem is striking a balance between being open to non-conventional ideas, and not being tolerant of non-sensical ones. I don't know Bem or his ideas, so can't comment on that side of things.
If the critics are right, and we also spotted a variety of problems in our quick perusal of the paper, then this is a case in which the peer review system failed.
This also reflects what happens to humans, in the past and even in our age of science, in the face of perplexing phenomena. There's a tendency to construct (make up?) possible explanations. When this is framed in reasonable hypotheses, then they can be addressed in acceptable ways.
But when the hypotheses are far out, we're less sound in handling them. The real problem arises when it isn't even clear that the phenomenon actually exists as something that demands explanation.
Calling crap by its true name is fine, but Galileo was put under house arrest as a symbol of the ostracism faced by heretical views. Even if only a tiny fraction of them turn out to be true, we should be open to what seem like oddball ideas.
In this case, I have no idea how the paper managed to be published. The putative explanation in terms of quantum theory or whatever was so post hoc that it's hard to see how it was persuasive to anyone.
There is (or was) a journal called Medical Hypotheses that was founded to serve as an outlet for the Twilight Zone ideas. Few people paid any attention. Should major journals have a section of Speculative Science? It would separate out such things from the kind of criticism Bem's paper gets and probably deserves. The papers probably should not have the usual sort of peer review....but should have some kind of review? Or does that just cater to and reward crackpots?
Maybe the reality is that these kinds of things properly have to struggle in the shadows of scorn until and unless they find some real support, which means support that a phenomenon actually exists in some definable, meaningful sense, AND that there is good evidence for it.
Science (and wishful-thinking) seems always to have its fringe, and some spokespersons who can get attention. Is it mainly harmless? That would be a hard case to make, depending on what counts as 'fringe'. But new ideas do eventually push through (continental drift is a kind of example, perhaps).
"Should major journals have a section of Speculative Science?"
I suppose that any scientific hypothesis that has yet to ascend to the status of theory is speculative science. Or would state this a different way?
Also, in the case of cosmology, any multiverse hypothesis is speculative.
In short, I don't see how we could develop criteria for "Speculative Science."
"Maybe the reality is that these kinds of things properly have to struggle in the shadows of scorn until and unless they find some real support, which means support that a phenomenon actually exists in some definable, meaningful sense, AND that there is good evidence for it."
I should also add that I completely agree with this.:)
Post a Comment