Why Science Rules (Even When it Sucks)

New Yorker Magazine published an article last month that on its surface seems like a resounding condemnation of science, the scientific method and the scientists who purport to follow it. It seems that in spite of the claims that science is neutral and objective, the pressures of money, fame and egotism still create bad science ... a lot of bad science. And we're not talking about trivial stuff. This affects people's lives, fortunes and health.

Author Jonah Lehrer's article The Truth Wears Off reveals some shockingly bad science:

• The drugs Abilify, Seroquel, and Zyprexa were shown to dramatically improve the condition of schizophrenics and are now heavily prescribed. But follow-up studies simply haven't been able to reproduce the original results. The drugs are effective, but not to the extent originally claimed.
• Zoologists showed that asymmetry in birds was strongly linked to the number of genetic mutations in the birds, and others went on to find similar results many other species including flies and even humans. But later studies weren't able to reproduce the results. It was as though the asymmetry effect had evaporated.
• Dozens of acupuncture studies in Japan and China, conducted by reputable scientists and reviewed by qualified peers, showed 100% positive results. But the same studies, when conducted by American scientists, were ambivalent; only 56% of the studies showed positive results.
• Out of 432 scientific claims about a link between genetics and a mental disease, only one was consistently replicable. The rest were junk science and wishful thinking.

The list goes on and on. It's enough to make reasonable people question and even reject science entirely.

According to Lehrer, the problem seems to be that science heavily favors positive results, to the point where science has trouble staying honest.

It starts with the scientists. There are no accolades in saying you didn't discover something. Nobody ever got famous for not discovering DNA, or not inventing a drug, or not figuring why people who leave a light on at night are nearsighted. Scientists get famous for positive, dramatic results.

And the scientific journals, in spite of their peer-review process, are heavily biased toward publishing exciting new results. When they publish a dull non-discovery nobody reads it, nobody cites the article, and nobody tries to replicate the dull results.

These two effects seem to be multiplicative. The scientists unconsciously bias their results to positive, exciting new discoveries, and the publications skim the very most fascinating and exciting discoveries off the top. Out of the millions of studies done each year by reputable, honest scientists, there are bound to be stasitical anomalies, errors and results that are confounded by unexpected side effects. But these are the very results that the positive-results bias pushes to the front.

But who is trying to fix this problem? Who is holding scientists' feet to the fire? Is it politicians? Is it the young-earth creationists or even the intelligent-design proponents who are always looking for the flaws in modern science? Is it the homeopathics and herbal remedy manufacturers? No, no and no. It is the scientists themselves.

This is where science itself comes to the rescue. Scientists understand that they have a problem, and they are the ones working hardest to fix it. As the great physicist Richard Feynman famously said:

"The first principle [of science] is that you must not fool yourself – and you are the easiest person to fool."

And that, in a nutshell, is what the scientists are now doing.

It's the scientists who discovered the problem, and scientists who will change their own culture to try to overcome the bias that they discovered. Because that's how science works. You can fool yourself and your colleagues for a while, but sooner or later the scientific method works. The bad data just don't add up and can't stand the test of time. The scientists take another look, discover why they were wrong the first time, and fix it. As Carl Sagan said:

In science it often happens that scientists say, 'You know that's a really good argument; my position is mistaken,' and then they actually change their minds and you never hear that old view from them again. ... It happens every day. I cannot recall the last time something like that happened in politics or religion.