The First Rule of the Science Fight Club: Fight

Matt Ridley is the best popular science writer in the business. I offer that observation as an example of the topic that Ridley concentrated on in three recent pieces for the Wall Street Journal: confirmation bias. While I can objectively defend Ridley's excellence, the main reason I rank him so highly is that he confirms a lot of what I already believe. Consider that mea culpa properly filed.

Ridley's first installment exposes a conceit of scientific culture.

There's a myth out there that has gained the status of a cliché: that scientists love proving themselves wrong, that the first thing they do after constructing a hypothesis is to try to falsify it. Professors tell students that this is the essence of science.

Yet most scientists behave very differently in practice. They not only become strongly attached to their own theories; they perpetually look for evidence that supports rather than challenges their theories. Like defense attorneys building a case, they collect confirming evidence.

He goes on to explain that this is "only human". Of course. The human mind is a narrative generator. Our thought and language consist of stories we construct out of the raw material that enters through our senses. Some of these stories are pragmatic. I am allergic to this food so I shouldn't eat it. Some of the stories are all about self-esteem. If they weren't in charge, I'd be doing better. Almost always the motives overlap. We persist in falling failed strategies because we can't admit to being wrong. Methods for rigorous analysis can help bring our narratives in line with reality, but they don't change the weight of the investments that we make in those narratives.

Confirmation bias occurs when we ignore evidence that is inconsistent with our narrative or theory in favor of evidence that confirms it. Despite their official loyalty to the principle of falsification, scientists are as much susceptible to confirmation bias as anyone else. So what corrects for this bias in the progress of science?

Ridley offers an answer in his second installment.

The answer was spelled out by the psychologist Raymond Nickerson of Tufts University in a paper written in 1998: "It is not so much the critical attitude that individual scientists have taken with respect to their own ideas that has given science the success it has enjoyed... but more the fact that individual scientists have been highly motivated to demonstrate that hypotheses that are held by some other scientist(s) are false."

Most scientists do not try to disprove their ideas; rivals do it for them. Only when those rivals fail is the theory bomb-proof. The physicist Robert Millikan, (who showed minor confirmation bias in his own work on the charge of the electron by omitting outlying observations that did not fit his hypothesis) devoted more than 10 years to trying to disprove Einstein¹s theory that light consisted of particles (photons). His failure convinced almost everybody but himself that Einstein was right.

This is obviously right. I say that because it confirms my bias concerns parties in politics. To make policy, like-minded persons have to join together, formulate positions, and work to see them translated into policy. A group of likeminded people is not, however, very open to evidence that tells against their biases. The solution is to have at least one honorable opposition.

Another reason that seems right is that it is not so much the best solution as the only solution. Once you come to grips with the reality of confirmation bias in science and everywhere else, what other solution is there?

In the final installment, Ridley turns to the specific problem of climate change.

Last month saw two media announcements of preliminary new papers on climate. One, by a team led by physicist Richard Muller of the University of California, Berkeley, concluded "the carbon dioxide curve gives a better match than anything else we've tried" for the (modest) 0.8 Celsius-degree rise in global average temperatures over land during the past half-century-less, if ocean is included. He may be right, but such curve-fitting reasoning is an example of confirmation bias. The other, by a team led by the meteorologist Anthony Watts, a skeptical gadfly, confirmed its view that the Muller team's numbers are too high-because "reported 1979-2008 U.S. temperature trends are spuriously doubled" by bad thermometer siting and unjustified "adjustments."

Much published research on the impact of climate change consists of confirmation bias by if-then modeling, but critics also see an increasing confusion between model outputs and observations. For example, in estimating how much warming is expected, the most recent report of the Intergovernmental Panel on Climate Change uses three methods, two based entirely on model simulations.

Perhaps you can see the problem. Computer simulations are useful not only for understanding phenomena but for producing results that can be reported to the press. Such models, however, inevitably build in assumptions about what is going on and these are frequently designed to confirm what the designer expects to find.

Ridley does not consider the influence of nonscientific biases, such as political opinions. In the climate change controversies, politics exercises an enormous gravitational force. Most, but not all, of those making a strong case for the anthropogenic global warming thesis and the need for global legislation are comfortable with the idea that governments should exercise more control over private economic activity. Most, but not all, of the critics are not.

The critics of global alarmism are no more immune to confirmation bias than the proponents. The problem with the debate so far has been the constant attempt by the latter to shut it down. We are constantly told that the consensus of climate scientists is that global warming is real and that big changes in our economies are necessary to correct it. Consensus, however, is more often the enemy of good science than its friend. Almost all moments of genuine progress in science happen when some critic successfully challenges the consensus.

I think that Ridley is right on all counts but that is surely subject to confirmation bias on my part. I note that this involves not only political bias but a personal delight at seeing the consensus undermined. Ridley thinks that good science happens when one scientist tries to tear a big hole in the theories of other scientists. The latter either emerge stronger or suffer a deserved collapse. Of course, his preference for an adversarial culture in science may itself be a product of confirmation bias. But again, what else is there?

Comments

[Bill Fleming]

Let's throw this one in the hopper and get your read. Thanks for your attention in advance, KB.
http://www.rollingstone.com/politics/news/global-warmings-terrifying-new-math-20120719

[Bill Fleming]

Also related to your post is the marvelous book, "Thinking Fast and Slow." If you've not read it, let me give it my highest endorsement. Put it on the top of your list. I'm 2/3 of the way through and the experience has been transformative. Kind of a "Thanks, I needed that" rush every 10 pages or so.

Enjoy:

http://www.psychologicalscience.org/index.php/publications/observer/obsonline/thinking-fast-and-slow.html

[Donald Pay]

The problem here is that Ridley looks at things from a vastly different scale than scientific study done by climate modeling have to be looked at. Hypotheis testing is done all the time on specific aspects that go into climate models. How various layers of the atmosphere act and interact, how chemical reactions in various layers of the ocean are affected by temperature, CO2, acidity, etc., etc. are mostly accomplished through hypothesis testing. But modeling of climate and climate change over time involves putting together various fields of study. It is a vastly complicated endeavor that has to take the best, if not always perfect, and most accepted knowledge from vastly different disciplines. There are arguments about how various parts of the model interact, and these are also subject to testing.

Then the output of the models are tested against predictions. Since these are usually long-term predictions, it is difficult to test them against future data, so data sets (hopefully independent of the sets used to develop the model) have to be used to try to "predict" past events. It is not a perfect situation, but there is no other way to do this science.

In the 1970s I worked on a project collecting data to test and refine a model of eutrophication in lakes. I always thought the modelers were just curve fitting, but over the years such predictive models have improved, and they've proved very useful in suggesting key drivers of eutrophication in many lakes, and what inputs (phosphorus in many instances) should be controlled for maximum benefit.

There will always be a tension between the scientists who work at a smaller scale and those who model at higher scales. Such was and is the case with all models, including the ones that the petroleum and uranium industry depend on to predict impacts to groundwater and production from their wells.

[Stan Gibilisco]

In my opinion, the people who want to expand government will latch onto any excuse for raising taxes or creating new taxes, so that they can get new revenue. Climate change offers a good option (carbon tax, gasoline tax, mileage tax, etc.).

I doubt that these big-government advocates really give more than one-quarter or, at the most, one-third of a hoot about the environment, climate, or welfare of the planet. They just want revenue. Period.

That said, I do believe that climate change is taking place, and I do believe that humans are contributing to it. As for whether that in itself is good or bad or indifferent, however, I cannot say. And as for what we can actually do about it ... I fear little to nothing.

And I cannot prove that my beliefs in fact represent reality. That task belongs to objective scientists, who can work without pressure from politicians, special interest groups, corporations, or any other force besides truth-seeking.

[A.I.]

Who are these critics of "global alarmism" of which you speak? My understanding is Richard Muller--who you fail to identify as a former global warming critic--was the last climate scientist of any standing to dispute anthropologic global warming and, by extension, the need to institute policies to counter it. Does his conversion not pretty much end the so-called debate?

[Donald Pay]

Interesting that you point out what you claim to be the political motivations of climate scientists, but never state the obvious—the skeptics, mostly non-scientists or scientists not actively doing research in any field related to climate, are funded by the fossil fuel industry, and supported by right-wing think tanks/blogosphere and public relations firms.

In my lifetime I've seen this pattern repeated over many major issues—the biologic effects of ionizing radiation, the health impacts of cigarettes, the impacts of DDT and other pesticides on wildlife and human health, the health impacts of agent orange on veterans and Vietnamese, chlorofluorocarbon impacts on the ozone layer, sulfur dioxide and nitrogen oxide emissions causing acid rain, fine particulate emissions causing respiratory problems. The deniers have a huge industry set up to create political hurdles and cast doubt on credible science. In every case, the deniers have been wrong. In every case the scientists were proven right. In every case people died because the deniers created regulatory delays. The people that do this and who pump this sort of pap to mislead people are murderers.

[Ken Blanchard]

A.I.: You crave an end to the debate. That is not an attitude friendly to science, as Ridley has shown.

[Ken Blanchard]

Donald: I do state the obvious and confirm it repeatedly by subjecting myself to the critique. Both sides have constructed enormous industries on the basis of their views. Whatever the motives of either side may be, dissent is good.

[indian silk sarees]

I will bookmark your web site and take the feeds also...I am happy to find numerous helpful information right here in the post