To understand why science may be reliable or unreliable, you have to recognize that science is done by human beings, and that how they interact with one another is absolutely crucial. Here's how I think scientific knowledge accumulates; I call it "the knowledge filter":
Science gets done through a communal process that's like the action of a large filter funnel with several stages, through which the murky mess of humans making many, often contradictory claims of truths about the world eventually yields a little trickle of fairly clear understanding.
Figure 1
The human urge to know and to convince others is a pandemonium of fantasies and folklore, hallucinations and religious cults, myth and pseudo-science, not just empirical and logical investigation. Just think of all the New-Age magazines and pop-religious paperbacks that flood the bookstores: dianetics and Scientology, extrasensory perception and reincarnation and channeling, astrology and Tarot, revelations from Nicholas Tesla and other neglected supposed geniuses. Nowadays, anyone who seriously wants to contribute to scientific knowledge had best not start there but rather get some undergraduate and graduate training in science: that narrows the mouth of the filter funnel, by educating about already established knowledge, about what's plausible and what isn't: Figure 2
Learning to do research means learning to ask yourself all the time, "What will others think about this?", because those others will examine you and decide whether you'll graduate or not, whether you'll be recommended for jobs and grants - or not, whether your papers will get published - or not, whether you get promotions and better jobs and prizes - or not.
Through that awareness that you have to satisfy the opinions of others, and through the actual practice of having colleagues and competitors look over grant proposals and manuscripts, much nonsense, pseudo-science, and stupidity is still-born, or at least filtered out before it's gotten very far.
So the ferment of scientific research, of frontier science, is a bit more disciplined than the general intellectual level of society as judged by what's on bookstore shelves. But research is still a rather messy business. Scientists differ in competence and in integrity; they're rebellious in varying degrees, toward established knowledge and toward established practices; they vary in creativity, interests, judgment, patience, and so forth.
But what they produce becomes a little less subjective, a little less imperfect by the time it gets published, because in order to get into print, it isn't enough to be personally convinced of some scientific fact - not even if you've become convinced by observing, setting up hypotheses, and testing them: you have to show evidence strong enough to persuade others that you're right, or at least that you're not obviously wrong. You have to produce evidence strong enough to convince people who start with different beliefs and prejudices than you do. So the primary literature of articles in research journals is more disciplined, more objective and less personal than what goes on in individual labs.
Still, the primary literature is anything but entirely consensual - there are competing theories and even competing, apparently contradictory results. As John Ziman has pointed out, the primary literature isn't scientific knowledge, it's merely information that certain claims have been made (10). If those seem interesting enough to others, they'll be used and thereby tested and perhaps modified or extended - or found to be untrue. Whatever survives as useful knowledge gets cited in other articles and eventually in review articles and monographs, the secondary literature which is considerably more consensual and reliable than the primary literature.
But still that's far from gospel. If after still more use and modification, including use by people in other specialties, if still no damaging flaws have turned up, then the knowledge is likely to get into textbooks. This textbook science is very reliable. It's been cleansed of most of the personal bias, error, and dishonesty that may have been there originally.
Yet even this textbook science isn't objectively true knowledge. Ziman guesses that while the primary literature in physics is perhaps 90% wrong, textbooks in physics are perhaps 90% right - by no means 100% right. The next century's textbooks of science will be significantly different from today's, even more than by the 10% that's wrong in today's, because there will not only be correction of errors but inclusion of things that we can't even dream of at present.
This knowledge filter illustrates that it's peer review, and the awareness of peer review, and the passage of time that makes scientific knowledge non-subjective and reliable. But there's nothing automatic about peer review or self-discipline. If peer review is cronyism - if scientists believe it proper to praise their friends and relatives rather than meritorious work irrespective of who does it - then false views and unreliable results will be disseminated.
Contrast this filtering with the popular notion of an "information explosion" that implies a crisis of coping with new knowledge; when rather it's a matter of weeding out from a mass of rubbish, a small amount of valid, useful, meaningful stuff. This model would suggest a different way of doing things than is now the generally accepted one. We seem to think that more research is always better, and that publishing original research is more worthy than writing review articles or books. But perhaps, given the mass of rubbish that needs filtering, perhaps less research would be better than more?! Maybe writing review articles and textbooks should be rewarded more than producing research articles?!
Version: 1.0, text updated: 5/12/1995
