Lousy science is epidemic and has been for a long time — so long few know what science actually looks like. That’s why I had to write this:
I’d be happy with just a modicum of honesty and transparency?
The media release is below.
###
Stanford researcher and colleagues announce master plan for better science
STANFORD UNIVERSITY MEDICAL CENTER
An international team of experts has produced a “manifesto” setting forth steps to improve the quality of scientific research.
“There is a way to perform good, reliable, credible, reproducible, trustworthy, useful science,” said John Ioannidis, MD, DSc, professor of medicine and of health research and policy at the Stanford University School of Medicine.
“We have ways to improve compared with what we’re doing currently, and there are lots of scientists and other stakeholders who are interested in doing this,” said Ioannidis, who is senior author of the article, which will be published Jan. 10 in the inaugural issue of Nature Human Behavior. The lead author is Marcus Munafò, PhD, professor of biological psychology at the University of Bristol.
What’s holding science back?
Each year, the U.S. government spends nearly $70 billion on nondefense research and development, including a budget of more than $30 billion for the National Institutes of Health. Yet research on how science is conducted — so-called meta-research — has made clear that a substantial number of published scientific papers fail to move science forward. One analysis, wrote the authors, estimated that as much as 85 percent of the biomedical research effort is wasted.
One reason for this is that scientists often find patterns in noisy data, the way we see whales or faces in the shapes of clouds. This effect is more likely when researchers apply hundreds or even thousands of different analyses to the same data set until statistically significant effects appear.
The manifesto suggests it’s not just scientists themselves who are responsible for improving the quality of science, but also other stakeholders, including research institutions, scientific journals, funders and regulatory agencies. All, said Ioannidis, have important roles to play.
“It’s a multiplicative effect,” he said, “so you have all of these players working together in the same direction.” If any one of the stakeholders doesn’t participate in creating incentives for transparency and reproducibility, he said, it makes it harder for everyone else to improve.
“Most of the changes that we propose in the manifesto are interrelated, and the stakeholders are connected as if by rubber bands. If you have one of them move, he or she may pull the others. At the same time, he or she may be restricted because others don’t move,” said Ioannidis, who is also co-director of the Meta-Research Innovation Center at Stanford.
Manifesto
The eight-page paper describing ways to improve science includes four major categories: methods, reporting and dissemination, reproducibility, and evaluation and incentives.
Methods could be improved, the authors reported, by designing studies to minimize bias — by blinding patients, doctors and other participants, and by registering the study design, outcome measures and analysis plan before the research begins — to prevent subsequent deviations from the study design, regardless of intriguing, serendipitous results.
The authors also state that reporting and dissemination might be improved by eliminating “the file drawer problem,” the tendency of researchers to publish results that are novel, statistically significant or supportive of a particular hypothesis, while not publishing other valid but less interesting results. “The consequence,” wrote the authors, “is that the published literature indicates stronger evidence for findings than exists in reality.”
The file drawer effect is fueled, though, from the behavior of universities, journals, reviewers and funding agencies — not just that of individual scientists, the authors write. One way funders and journals can help is by requiring all researchers to meet certain standards. For example, the Cure Huntington Disease Initiative has created an independent standing committee to evaluate proposals and provide disinterested advice to grantees on experimental design and statistical analysis. This committee doesn’t just set standards; it actually helps researchers meet those standards.
The ultimate goal is to get to the truth, Ioannidis said. “When we are doing science, we are trying to arrive at the truth. In many disciplines, we want that truth to translate into something that works. But if it’s not true, it’s not going to speed up computer software, it’s not going to save lives and it’s not going to improve quality of life.”
He said the goal of the manifesto is to increase the speed at which researchers get closer to the truth. “All these measures are intended to expedite the process of validation — the circle of generating, testing and validating or refuting hypotheses in the scientific machine.”
###
Researchers from the University of Virginia; the University of Oxford; the University of Bath; Cardiff University; the National Centre for the Replacement, Refinement and Reduction of Animals in Research; the Wharton School; and the Cure Huntington Disease Initiative Foundation also co-authored the study.
Funding was provided by the British Heart Foundation; the Cancer Research U.K.; the Economic and Social Research Council; the Medical Research Council; the National Institute for Health Research, under the auspices of the U.K. Clinical Research Collaboration; a Wellcome Trust Principal Research Fellowship; an unrestricted gift from S. O’Donnell and B. O’Donnell to the Stanford Prevention Research Center; and a grant by the Laura and John Arnold Foundation to the Meta-Research Innovation Center at Stanford.
The Stanford University School of Medicine consistently ranks among the nation’s top medical schools, integrating research, medical education, patient care and community service. For more news about the school, please visit http://med.stanford.edu/school.html. The medical school is part of Stanford Medicine, which includes Stanford Health Care and Stanford Children’s Health. For information about all three, please visit http://med.stanford.edu.
” how many of these anomalies are being factored into our surface temperatures?” No way of telling, but if you consider the fact that the high for a day generally is not reached until late afternoon, and, especially in the late fall and winter months, in the northerly climates such as much of the US the temperature often rapidly drops once the sun goes down and cooler/cold temperatures prevail for the next 14-16 hours before slowly rising to the next day’s high. Thus if one simply averages the high and the low for the “average” temperatures (as is done by our local paper) for an area, it is significantly biased on the high side, which suits our paper’s editorial policy perfectly.
Something was called to my attention yesterday re/ temperatures. We had an unusual “bungey cord” high. All day long the temperature hung around 30 degrees, but after dark it suddenly shot up to 53 degrees for about 20 minutes, then dropped back down into the low 20’s. Now the official high is listed as 53, but the actual daytime temperature (that lasted all day long) is only 30. Nor is this unusual, a high may only last for less than an hour before dropping down 10 or 15 degrees. I realize this can happen to cold temperatures as well, but how many of these anomalies are being factored into our surface temperatures?
Eisenhower’s farewell speech is constantly cited for the term “military-industrial complex.”
But in that same speech he also warned that government funding of science would control scientific thought and eliminate intellectual curiosity. He would not be stunned by the government control of global-warming science, and related fields–this is exactly what he warned against.
Junk science will be around in great quantity as long as politics are around to drive it. Don’t agree? Just follow the money for all the “popular” research that many too many “top” scientists (both real and self-designated” publish in politically motivated alleged scientific journals.