Bounce From Calvin Beisner–More on Epidemiology IV

I got the bounce from a guy who is interested in scientific integrity and knowledgable.

Many years ago I met Calvin Beisner and I may have introduced him as a speaker or one of the guys in his organization.
They have a religious point of view in regards to the human effects of climate science. I appreciate that.
So Cal is interested in statistical analysis that has integrity. Joe Bast mentioned that I have a thing about statistical significance and confidence. Actually, sort of.
Here’s Cal’s note to Joe about the Nature article summaries.
Thanks for all these, Joe. Very helpful summaries.
Regarding the piece on statistical significance, there is a major critique of “p-value” that you might want to recommend to your distribution list members: Stephen T. Ziliak and Deirdre N. McCloskey, The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives (University of Michigan Press, 2008, 352 pages).
Regarding the maladies of peer review, see my “If Peer Review Were a Drug, It Wouldn’t Get on the Market,” which cites a number of studies indicating that peer review doesn’t significantly increase the reliability of published articles, and my “In Praise of Scientific Piranhas: the Legacies of Climategate” and “Wanted for Premeditated Murder: How Post-Normal ‘Science’ Stabbed Real Science in the Back on the Way to the Illusion of ‘Scientific Consensus’ on Global Warming,” both of which also show some of address the weaknesses of peer review and its tendency to contribute to herd thinking rather than independent, skeptical thinking.
In Christ,
Cal
E. Calvin Beisner, Ph.D., Founder and National Spokesman
Cornwall Alliance for the Stewardship of Creation
9302-C Old Keene Mill Rd., Burke, VA 22015, Phone 703-569-4653; Cell 954-547-5370
From John:
I am a biologist and hate numbers, but I think numbers are important when populations are being studied so you don’t follow your biases.
So here’s a couple of things that are important.
Power–power of a study is how big and robust is the study to get you in the range where you can reduce the problem of randomness and the unreliability of small studies.
When your power is good then you can get p values that are good, which is a measure of how much randomness you are dealing with, so if the p value is 0.05 that means you have a 1 in 20 chance of randomness causing you to have an error–false positive is a type I error, false negative is a type II error. When the p value is 0.05 or less the results are pronounced Statistically Significant.
Statistical Significance, as I might suppose Calvin would describe it–is misused, because authors with small associations that don’t prove anything, dress their results up as statistically significant, which is only a test to reduce the chance of randomness.
Confidence Interval (CI) is the range of confidence that is calculated for the result of the study, and in toxicological studies of populations, the result is often given in Relative Risk, for example reported as RR is 1.3, CI 1.19-1.42 (odds ratio and hazard ratio are used too, but similar). When the result is in a Relative Risk that is small, less than 2.0 then I know the study fails to have evidence that proves causation in a toxicological study because tox requires a robust effect in the exposed population–a small RR only suggests a possible effect so further studies would be required and no study with a Relative Risk of less than 2.0 should be considered good evidence for causation. That’s important, don’t you know.
It is also important to know that a pile of studies with a small no proof RR does not add up to evidence of proof. That is a fallacy that is well-known in science. A pile of weak evidence is still weak.
One more than that is essential in this struggle about proof of toxicological effect is that if i have a study that disproves the effect that negative study trumps or outweighs a small effect study and also has great weight on disproving a hypothesis (as Einstein said, one experiment can extinguish the most elegant theory).
To avoid getting Stan Young all worked up this late in the evening, I will fore go the discussion about multiple inquiry and multiple endpoints that get’s into proper adjustments for p values and confidence intervals. The reason is that the risk of random false positives and negatives increases when you look for multiple effects and ask multiple questions in a group study that has enough power to make p values good (0.05 or below) all the time.
The Adjustment is called the Bonferroni Claw, and Stan has asked epidemiology journal editors why they don’t require the adjustment and they have no good excuse.
My simple answer is it would screw up the epidemiology game they have going and put journals and researchers out of business–some studies make things out of multiple inquiry, some small associations, some use both tricks and they can grind and torture the data inside the parameters of the game to find the answers that they seek–driven by intellectual passion, tunnel vision and confirmation bias, fueled by ambition. As Feynman said–you first, as a good scientist, have to be your own most careful and objective critic. your own most skeptical observer. Humility is essential to good science.
Stan can explain it, I just get mad about it. Small associations dressed up in statistical significance are still JUNNNNNKYYY.