“Research spin is when findings are made to look stronger or more positive than is justified by the study.”
Hilda Bastian writes at PLOS.org:
2 thoughts on “How to Spot Research Spin: The Case of the Not-So-Simple Abstract”
Comments are closed.
“Research spin is when findings are made to look stronger or more positive than is justified by the study.”
Hilda Bastian writes at PLOS.org:
Comments are closed.
Sensible. A few more rules, mainly for observational studies. If the study and its claim are important to you, then
1. Count the number of questions at issue. How many questions can be at issue. What other papers could they have written?
2. Look at the p-value for their claim. Multiply that p-value by the number of questions at issue. If that new p-value is not less than 0.05, forget about their claim.
As most claims from observational studies do not replicate, a simpler rule is to just ignore them.
Or, in other words, they lie. But “spin” is a nicer more genteel description of what they are doing. I have mentioned before that my book on cholesterol included an appendix titled “How to Read a Medical Journal Report”. I would reproduce it here but it runs about four to five pages and would be a little too long for a comment. But here are a few quotes to give you some idea how I analyze a medical report.
“1) Summary or Abstract: Read this to get an overview of what is to come, but don’t take it too seriously, as it often reflects the subjective impressions of the authors.”
“2) Introduction: Approach this the same way as the Summary or Abstract. The authors will often include matter-of-fact statements about conclusions or theories that are highly debatable. These are often self-serving and should be taken with a grain of salt.”
“Authors will almost always list a “p” value for each end point. P values are supposed to tell you whether a difference found between two groups is statistically significant. You should ignore these and simply compare the differences in absolute risk and use common sense to tell you if the differences are of any practical significance.”
“Any study that doesn’t give the actual numbers for the raw data and absolute risk can be dismissed out of hand. Some reports will give the raw data in tabular form but never mention it in the text of the article. Rather, the authors will present a variety of derivative statistical measures such as risk ratios and complicated regression formulas. Ignore all of these and seek out the basic numbers. Assume that if the basic numbers are not given, the authors are hiding something.”