Comments deserving a special post.

I am just a moderator, and a slug science guy.

These comments are exceptional and deserve some more airing.

Thanks to the writers.

marque2 | June 7, 2014 at 2:18 pm | Reply | Edit
People confuse Peer review, with independently reproduced. Peer review, just makes sure your math is So so (not even good, as the reviewer probably isn’t a mathematician).

What is really needed is for an independent set of scientists to independently reproduce the results. this isn’t done anymore, because of budget, and because scientists know about the 1/20 false positive rate, which is probably responsible for a lot of papers, and then there are the social aspects, don’t really want to disprove the hot scientists results, and then there are the nefarious aspects, don’t reproduce them or they will reproduce you, and it might just be discovered that your whole field is based on false premises. Can’t have any of that – so no one does it any more.

Bill | June 6, 2014 at 9:04 pm | Reply | Edit
All true, except you left out the initial short literature review, a null hypothesis, methods, probabilities, and a long list of citations, including those in support and opposition of the findings.

6 responses to “Comments deserving a special post.

  1. Coach Springer

    We don’t have time for all that when we’ve got TV shows to do and governments to run. (And Scientific American to print.)

  2. The author expresses views that unfortunately are widely shared. The reproduction of a finding is the ultimate peer review. In our book “Peer Review and Scientific Assessment” we discuss the requirements of independent peer review and identify shortcomings of the peer review process as used by a large number of scientific journals. Peer review, if properly performed confirms the claims of the author. In our book we compare the reliability of peer review with that of jury of peers. In both cases, even if every requirement is met the results are not perfect. In addition, peer review is not necessarily capable of detecting falsification of the data and results. Peer review is not perfect but is the only way to have a minimum level of acceptability of scientific claims.
    A. Alan Moghissi

  3. i am a peer reviewer for two journals.

    The work is very basic–evaluate the premise and the methodology, but there are no easy methods for testing the validity of the results or the methodology related to the research.

    There is a lot of uncertainty in the work that is published, so the best test is reproducibility.

    Verification. No simple answer other than reproduce the results.

  4. Maybe a simple disclaimer that says the results of this, Fill in the blank, have not been reproduced as of the date of publication and as such should not be considered to be the basis for any new field of research until it has been verified.

  5. These problems are PRECISELY the reason journals should not publish anything without full datasets and methodologies available (including computer code). No one can reproduce the results without those details, just ask M&M about the Mann’s hockeystick graphs.

    One should not have to threaten lawsuits or FOIAs to get data for already published material.

  6. In my work, we don’t make the underlying data used in our published studies available on-line. Mainly because our web site is hosted and thus, space limited. We don’t use grant money so there’s no requirement to make it publicly available.

    With the above said, we do offer to provide our data, formulas, and computer code upon request. We are required to by the journals to do so.

    It’s a shame that climatologists hide behind intellectual property claims and slack journal policies that don’t require the data and methods be made available.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s