Last week's issue of "The Economist" (Oct 19, 2013) had not one, but two articles dealing with the problems of modern science. "How science goes wrong" (p. 13) discusses how the the central premise of all modern science--the ability to replicate an experiment and verify its results--is largely a myth. The second article, "Trouble in the lab" (p. 26-30) goes even further. It first points out that this premise is the foundation of all modern science:
"The idea that the same experiments always get the same results, no matter who performs them, is one of the cornerstones of science's claim to objective truth."
However, it goes on to explain that this cornerstone premise is fundamentally flawed. Venture capitalists, it explains, have a rule of thumb that fully half of all published research papers cannot be replicated. But they may in fact be optimists-- when a drug company recently tried to replicate 53 landmark experiments in published cancer research, only 7 of the 53 results (13%) could be verified. The article goes on to point out and discuss several problems with scientific research as it is performed today, including:
- Poor experiment design;
- Improper statistical analysis;
- 26% of scientists know of other scientists that have falsified results;
- "Publish or perish" pressure on academics;
- Inadequate pre-publication peer review;
- Inadequate post-publication comment, and (where appropriate) withdrawal;
- Overwhelming editorial bias (95%+) toward hypothesis confirmation;
- Editorial bias toward surprising or newsworthy results; and
- Biased government and NGO funding mechanisms.
However, there are a number of other problems that they did not discuss. For example, the premise of most scientific experimentation is that you try to reduce all the variables except the one being observed. With the human body, that assumption is patently false. Particles, atoms and molecules may perform in readily predictable ways (with a little randomness thrown in), but living things often do not. The problem becomes even more extreme when the experiment involves herbs, which can vary in chemical makeup depending on time of harvest, seasonal weather patterns, habitat, and myriad other factors. Add to that the variations across several different herbs in a complex formula, and you really have a lot of variables to try to control.
The confirmation bias in the human mind (which is the essential tool of all science) is an area that is just beginning to be explored. This subject goes far beyond the publication biases discusseed in the Economist articles-- the idea is that the mind tends to find relationships in data, whether they exist or not, and marshals the data to support the relationship. This may in part explain the confirmation bias in scientific literature.
In addition, the costs of clinical trials are driven by well-capitalized drug companies with proprietary and patented products. These companies keep clinical trial prices very high, and increasing all the time. Physicians with clinical research resources prefer drug trials because they pay better, are more visible to the medical community to which they belong, and offer higher chances of publication. Of course the high costs of clinical trials increases the temptation to "cherry-pick" or even to falsify results, and even large drug companies have admitted that such problems are widespread.
All this is not to say that science isn't important-- it is. But all that looks scientific is not science. The fundamental principles of scientific research must be observed, no matter who is funding or publishing it. Big names supporting the research may be nothing but expensive corporate window-dressing for enhanced credibiltiy and sales.
In many cases, a clinical trial or two, even if well-designed and well-executed, may not add very much to what is already known about an herb or herbal formula, especially if it has a long-standing written history of use on a wide variety of people. Such an herb is often just as effective, and much safer, than a proprietary drug with a couple of "cherry-picked" gold-standard clinical trials, as the recent rash of drug recalls has clearly shown. In this, as in most areas of life, there is just no substitute for doing the homework.