In science we have built-in self-correcting machinery. Strict double-blind controls are required in experiments, in which neither the subjects nor the experimenters know the experimental conditions during the data-collection phase. Results are vetted at professional conferences and in peer-reviewed journals. Research must be replicated in other laboratories unaffiliated with the original researcher. Disconfirmatory evidence, as well as contradictory interpretations of the data, must be included in the paper. Colleagues are rewarded for being skeptical. Extraordinary claims require extraordinary evidence.
We need similar controls for the confirmation bias in the arenas of law, business and politics. Judges and lawyers should call one another on the practice of mining data selectively to bolster an argument and warn juries about the confirmation bias. CEOs should assess critically the enthusiastic recommendations of their VPs and demand to see contradictory evidence and alternative evaluations of the same plan. Politicians need a stronger peer-review system that goes beyond the churlish opprobrium of the campaign trail, and I would love to see a political debate in which the candidates were required to make the opposite case.
Skepticism is the antidote for the confirmation bias.
