Correlations are not causations — except at universities and in public policy?

Beginning with a false premise, university researchers dredged computer simulation studies looking at associations (risk factors) and reported that one of the variables  could be causal. This, it appears, now passes for science and support for public policies.

Research in support of politically correct foods are undergoing the same sorts of computer modeling machinations as research showing people cause global climate change. Reporting from the press release, MSN told the public:

Junk food taxes pay off, study finds

Taxing soft drinks and foods high in saturated fats and providing subsidies for fruits and vegetables might encourage people to change their eating habits and possibly improve their health, according to a new study. Researchers in New Zealand analyzed 32 previous studies and concluded that there would be a 0.02 percent decline in consumption of fatty foods with each 1 percent price increase. They also determined that a 10 percent increase in the price of soft drinks would decrease consumption by between 1 percent and 24 percent….

This suggests that such food pricing strategies have the potential to reduce dietary inequalities, said Helen Eyles and colleagues from the University of Auckland and the University of Otago, in Wellington, in a journal news release. Diets high in sugar and saturated fats contribute to the development of chronic diseases, such as cardiovascular disease and diabetes.

Their article, A Systematic Review of Simulation Studies, was published in PLOS Medicine. The researchers began by explaining their premise, one also popular in public policies: junk foods, with high saturated fats and salt content, are “causative risk factors” for chronic diseases, including heart disease, strokes, diabetes, cancers and respiratory diseases. Reducing population diets of these foods, they noted, were identified as a priority action by the United Nations during its meeting on prevention and control of non-communicable diseases. “If these changes in population diet take place, the interventions will support the global goal of reducing non-communicable death rates and averting tens of millions of premature deaths within the next decade.”

So, they did a literature search for computer simulation studies published since 1990 that looked at the association between food pricing strategies, consumption and chronic diseases. They found “19 peer-reviewed papers and 13 other types of reports” to use in their analysis.

But, they admitted the majority of the included studies (27/32) were of low to moderate quality. According to the authors, there was also “substantial variability in model structures, data inputs and the types and magnitudes of food taxes and subsidies assessed.”

  • They weren’t able to use meta-analysis techniques as are used with randomized controlled trials to estimate the effect of an intervention, they said, since they were combining findings from simulations and models using varying structures and mathematical techniques. So, a key aspect of their study, they said, came from synthesizing epidemiological models and estimates of price elasticity.
  • They also admitted that none of the studies included in their review had attempted to validate the epidemiological model that had been used, even though “validation is important because underlying model structure and assumptions vary widely between models and are associated with uncertainty; without validation or comparison of findings with other models, it is difficult to determine whether findings are real, or in fact artefacts of the model itself.”
  • Furthermore: “Most studies in this review (25/32; 78%) failed to estimate the uncertainty of model findings. Uncertainty arises from the model structure and variation in the model inputs, including food consumption data, food prices, relative risks, and PEs.”

Despite their findings that “suggests food pricing strategies have potential for changing population diets and long-term health and disease outcomes,” they wrote, “high-quality evidence is lacking, particularly with regard to the unintended effects of compensatory purchasing and the potential impacts on health equity, long-term health, and NCD mortality.”

In fact, they highlighted:

There was also some evidence that pricing strategies may result in unintended compensatory buying through cross-PEs; two moderately high quality studies estimated a potential increase in consumption of sodium in response to a saturated fat tax, and a potential increase in mortality from CVD in response to a tax on less healthy foods.

In this paper, the authors explained they did little more than pool estimates from simulated models:

A particular aim of this review was to collate and quantitatively summarise the best evidence from simulation modelling studies regarding the association between food pricing strategies, food consumption, health, and NCDs. Therefore, although the pooled estimates are based on lower quality studies, the estimates can be improved upon as more relevant, higher quality research becomes available…

Nevertheless, a correlation found in simulation models was suggested to have a causal effect:

Notwithstanding the low to moderate quality of the majority (27/32) of the included studies, the overall finding of this review is that pricing strategies have the potential to produce changes in population food consumption….

Based on modelling studies, taxes on carbonated drinks and saturated fat and subsidies on fruits and vegetables would be associated with beneficial dietary change, with the potential for improved health.

But they did admit the need for more research:

Robust evaluations built into the implementation of food pricing policies would help to answer some of these questions and engender confidence that such strategies will provide positive effects…

Of course, in the real world, fat taxes have done little to affect obesity rates or health outcomes. “While we should not expect large health benefits from fat or sugar taxes, the administrative costs are real and substantial,” Dr Eric Crampton, Senior Lecturer in Economics, University of Canterbury, told the New Zealand Science Media Centre.

7 responses to “Correlations are not causations — except at universities and in public policy?

  1. GIGO!

  2. the below would explain how the mountain of JUNK SCIENCE on shs/ets has been made,especially on the cheap!

    Home » 1a – Epidemiology.The design, applications, strengths and weaknesses of descriptive studies and ecological studies
    Descriptive studies, sometimes known as geographical or ecological studies, can be used to demonstrate patterns of disease and associated factors in a population. The units of study are populations or groups

    Reasons for the ecological fallacy include:

    •It is not possible to link exposure with disease in individuals – those with disease may not be the same people in the population who are exposed
    •Data used in descriptive studies were usually collected for other purposes originally
    •Use of average exposure levels may mask more complicated relationships with the disease
    •Inability to control for confounding
    Strengths of ecological studies

    ?Cheap and simple to conduct.
    ?Utilize routinely collected health statistics.
    ?Exposure data often only available at area level.
    ?Differences in exposure between areas may be bigger than at the individual level.
    ?Utilize geographical information systems to examine spatial framework of disease and exposure.
    ?Generate hypotheses to examine at the individual level.
    Weaknesses of ecological studies

    ?Measures of exposure are only a proxy based on the average in the population. Caution needed when applying grouped results to the individual level (ecological fallacy).
    ?Potential for systematic differences between areas in recording disease frequency. For example there may be differences in disease coding and classification, diagnosis and completeness of reporting.
    ?Potential for systematic differences between areas in the measurement of exposures.
    ?Lack of available data on confounding factors.

  3. Silly and tragic. There is a 100% correlation (r2 = 1) between consumption of water and death.

  4. Honey dippers are back.

    For those too young to remember, honey dippers were guys who cleaned out the human waste under outhouses.

  5. The “new science” tends to be more about “consensus” than fact, and computer models are great at generate very good outcomes that will gain consensus, and people can pear review the data and consent to the outcomes

  6. “But they did admit the need for more research”. I.e., trolling for research funding. This isn’t science, it’s a Request For Proposals (RFP). Point out the weaknesses of other research in the field and imply ‘we’re the ones who know how to do it right’. Sometimes it’s hard to distinguish between scientists and pimps.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s