Sunday, August 13, 2006

Seeing what we want to see

A Newsweek magazine poll some years ago found that 87% of Americans believe in a God who answers prayers. More than three-quarters of Americans believe God sometimes intervenes to heal someone with an incurable disease, and nearly as many believe prayers are answered for such mundane requests as finding a job. What is true for Americans is undoubtedly true for believers of every faith right around the world. If so many are convinced of the efficacy of petitionary prayer, how can they possibly be wrong?

Four centuries ago, Francis Bacon said that what a person would like to be true, he preferentially believes. Even the most fair-minded observer can be led into error by unconscious or unexamined prejudices, which is why scientists place so much emphasis on controlled experiments, the statistical analysis of data, peer review, mathematics, diagrams, photographs, specialized languages, and the strict exclusion of personal, religious and political affiliations from scientific communication. The point of all these strategies is to minimize the effect of "seeing what we want to see."

But no knowledge system, not even science, can be entirely free of personal and cultural predispositions. The history of science is full of wrong turns taken on the basis of personal or cultural prejudices. A subtle example of how easy it is to misjudge evidence is the work of the astronomer Adriaan van Maanen on the rotation of galaxies.

In the second decade of the last century, one of the biggest unanswered questions in astronomy was the distance to the so-called "spiral nebula", pinwheel-shaped whirls of stars and gas that could be observed with telescopes. Were the spirals among the stars of the Milky Way Galaxy, and therefore relatively nearby and small, or were they far beyond the Milky Way and perhaps as large as our own galaxy? One way to answer this question was to see if rotational motion could be detected in the spirals. If the spirals are small and nearby, they might appear to rotate more quickly than if they are larger and far away.

Van Maanen looked for rotation by comparing photographs of face-on spirals taken some years apart. It was a simple and apparently foolproof task he undertook: compare the positions of stars in two photographs of the same object. In spiral after spiral he found consistent evidence of rotation, as if the spiral nebulas were unwinding. (The images show the relatively nearby spiral galaxy M33 and van Maanen's presumed motions of individual stars in that galaxy from his 1921 paper.) The star displacements he observed were tiny but apparently real. Since van Maanen was one of the most respected astronomers in the world for this sort of work, his results were widely accepted. His measurements suggested that the spiral nebula were inside the Milky Way.

It took more than a decade for astronomers to realize that van Maanen's consistent and convincing measurements were in error. Others who attempted to reproduce his work were not able to do so. And other lines of evidence suggested that the spirals were outside of the Milky Way. In fact, we now know that the spirals are so far away that it would have been impossible to detect a change in the positions of stars during the intervals used by van Maanen.

So what was the source of his error? Historians Richard Berendzen, Richard Hart and Daniel Seeley have examined the van Maanen's work and rule out systematic instrumental and computational errors. It seems that van Maanen had a slight personal bias toward believing that the spirals were in rotation, and his results reflected this bias. In other words, while striving for complete objectivity, his measurements were nonetheless affected by his expectations.

The machinery of scientific knowing is designed to prevent or detect exactly this kind of error, and eventually did so in the case of van Maanen's galaxies. The arbiter of scientific truth is repeatable, reproducible experiment. Nature, not our hopes or desires, must have the last word.

One should always be skeptical about experimental results that lie close to the margin of systematic instrumental, statistical or personal errors. All of the so-called evidence for the efficacy of petitionary prayer falls into this category, and that is why every claim for answered prayers should be considered skeptically. History has taught us how easy it is to see what we want to see.

Further Reading

If personal and cultural factors influence what we see and how we interpret what we see, can we have confidence in science as an objective way of knowing? The historian of science Michael Ruse ponders this question in the context of evolutionary theory in Mystery of Mystery: Is Evolution a Social Construction? (Harvard University Press, 1999). Ruse considers the life and work of major players in evolutionary theory, from Charles Darwin to Richard Dawkins. Is there a real world out there? Can we know it? With what degree of objectivity? Ruse is modest in his conclusions, but they will be satisfying for scientists.

Discuss this essay and more over on the Science Musings Blog.