Here’s the best and most important read for today.Please don’t skip it.
I cannot add much to this in depth article on the unreliable nature of research.
However please pay attention to sections on powering studies, statistical significance and the sacred concept in science of replication of experimental studies and why so many “important” new studies are rife with error and can’t be replicated. Big problems with bad methods, tunnel vision, confirmation bias, chasing funding and irresponsible journal editors chasing the publication bias for new and important findings.
http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble
The idea that peer review should include other disciplines has been tossed around for quite a while. The big argument against it is that outside disciplines aren’t expert enough to review the work in other fields. But as discussed in this article, many of the mistakes arise from statistical and methodology errors. I believe that disinterested parties would be more critical of the methodology and analysis than fellow peers.
As discussed with climatology, there is a communal mindset within climate science. The foremost being that global warming is primarily a result of human added carbon dioxide. If you look, virtually all research starts with this premise as a given. Climatologists are unlikely to question this precept, as they are also using it as a basis for their research. So almost no climatologist at this point even does research to reaffirm this basic premise. So if this premise is wrong (as most of us non-researchers here think), there is no self-correcting going on at all.
I’ve only touched on a fraction of the problem peer-review with climate science. As mentioned, there are issues with pleasing the grantor, self-perpetuation, egos, and a host of other issues that plague the climate community that would certainly be helped at least somewhat, if the research were allowed to be scrutinized for “good” methodology and analysis by outside groups.
I’m sorry, but is the entirety of quantum mechanics, string theory, electronics, material science, chemical development (with plastics going from bakelite to thousands of different types with completely different properties), engine development, etc, etc, etc. Nothing? Or does it simply not conform to your pessimism?
We’ve made NO progress since b4 WW2 when all the scientists’ agreed that our number one goal should be to geld the feeble minded
When science becomes servant to ideology, all integrity, honesty and adherence to scientific rules is sacrificed to the ideology.
I have no claim to being a “scientist” , what ever that turns out to mean,
But my simple mind was schooled by reading and reading about Richard Feynman and was tempered in the forge of my readings about by and about Alan Sokal.
The summary of my beliefs is this:
The the inventor scientists look a something that has never been described before, and describes it as best he or she can.
The research scientists look at THAT and either try to expand the understanding, or prove that the original ideas were wrong.
The sophists come up with a marketable (in terms of buying tenure or a book deal) notion and write about it.
In order to get the idea past peer review they have to prove in the foot notes that somebody else credible has already said it, what ever “it” is.
Seems like team A produce paper. Team B produce paper saying they can’t reproduce A’s results. A calls B deniers. B calls A deniers.
I was not surprised at the condemnation of Peer Review. All of us know that peer review is at best an imperfect procedure, and at worst a rubber stamp. I was surprised at the statistics relative to type I and type II errors and what they mean to the publication of results that are wrong. It almost seems like what we need is a multi-layer peer review process that uses not just “the usual suspects” to review their colleagues’ papers, but also experts in methods, statistics, and logic.