The EPA's Secret Science

By Steven Milloy
February 2, 2001, FoxNews.com

“Independent Institute Finds Key Pollution Studies Are Sound Science,” blared a recent press release from the American Lung Association.

Yet one of the most controversial “junk science” issues of recent years — whether air pollution causes premature death — will remain controversial for the foreseeable future despite activist claims and media headlines.

The ALA goes on to say studies show “once again that air pollution shortens lives, and strong [Environmental Protection Agency] standards are needed to protect public health.”

But the reality is somewhat different — at least according to Health Effects Institute, the organization whose research the ALA cited.

The saga started in November 1996 when the EPA proposed more stringent regulation of airborne particulate matter, otherwise known as soot. The EPA initially claimed further regulation would prevent 20,000 “premature deaths” every year at an annual cost of $8 billion. By valuing each life saved at about $5 million, the EPA estimated the monetary benefits of saving these lives at $100 billion per year.

But the validity of the EPA’s estimates is in question. In particular, the estimate of premature deaths was based on the “Pope study,” so called for its lead researcher, C. Arden Pope of Brigham Young University. The Pope study reported that airborne particulates were associated with a 17 percent increase in premature deaths. But this result constituted only a weak statistical correlation — not scientific proof of a cause-and-effect relationship between air particulate pollution and premature death.

Although the study included more than 550,000 people, the researchers did not measure the level of exposure to air pollution for even one study subject. Instead, they guessed how much pollution these individuals might have encountered. The researchers also failed to look at the subjects’ diets, income, health histories, genetic predispositions to illness, exercise habits and social habits — all well-established risk factors for premature death.

The study did adjust for some factors — including smoking habits, education level and occupational exposures — but additional adjustments could easily negate the purported 17 percent increase in risk.

A further problem was that no one has ever demonstrated how typical levels of airborne particulates could cause premature death.

In this context, the reported correlation could easily have been a statistical artifact. According to the National Cancer Institute, “In epidemiologic research, [risks of less than 100 percent] are considered small and usually difficult to interpret. Such increases may be due to chance, statistical bias or effects of confounding factors that are sometimes not evident.”

But the EPA did not back down — even when an elementary statistical error in the EPA’s calculations was discovered that knocked down the lives-saved estimated to 15,000 or when some economists estimated the proposal’s cost to exceed $100 billion annually.

The controversy prompted Congress to ask the EPA to produce the Pope study’s raw data so independent scientists could check the results.

The EPA initially balked, saying there was no purpose in any re-analysis of the EPA-funded study. Finally, after issuing the regulations, the EPA did provide access to the data — to the Health Effects Institute, a Massachusetts-based research organization funded by the EPA and the auto industry.

HEI issued its report last week, three years after the regulations were finalized. The results are not quite as the ALA press release touts.

Using essentially the identical methodology as the Pope study researchers, HEI produced virtually the same results — hardly a surprise. A naive media reported this “replication” in headlines such as “Research on Air Particles Passes Muster” and “Studies Back Particulates’ Link to Death.”

But the re-analysis has essentially the same shortcomings as the original study. The researchers did not factor into their analysis the effect of diet or genetics on death rates. Data on exercise habits were factored in, and ended up reducing risk estimates by almost 30 percent. But the quality of the data on exercise is debatable; incredibly, it indicates no difference in death rates between non-exercising study subjects and heavy-exercising study subjects. Better quality exercise data could have an even more dramatic effect on the numbers.

There is a similar data-quality problem with health history. The re-analysis paradoxically indicates that healthy study subjects had twice the risk of premature death of diseased study subjects.

Neither of these data problems is surprising given how the study data were collected. The American Cancer Society amassed the data by having 70,000 volunteers — not trained data-collection specialists — go to their friends, family and neighbors and ask personal, health-related questions. No effort was made to verify or validate the study subjects’ responses.

It’s no wonder the HEI concluded its report: “It is important to bear in mind that the results of our re-analysis alone are insufficient to identify a causal relation with mortality.”

So why did the ALA jump to its conclusion? That question perplexed HEI’s president Daniel Greenbaum, who noted the ALA issued a press release last April titled “New Health Research ‘Vindicates’ EPA; Soot Particles Are Deadly, Lung Association Notes” — three months before the HEI report was made available to the public.

The explanation, though, lies with the finances and politics of the ALA. As first reported by Investors Business Daily in January 1997, “The ALA has had a long — and lucrative — relationship with the EPA.” In the years before the EPA air-pollution proposal, the agency gave the ALA almost $5 million — despite the ALA suing the EPA almost every year claiming the agency wasn’t complying with the nation’s air-pollution laws.

“If you think the EPA is upset with the ALA suing them, think again,” said Scott Segal, a Washington, D.C.-based attorney. “Truth be known, the EPA wants to be sued, because every time they are sued it expands the reach of the Clean Air Act.”

Unfortunately for the ALA and the EPA, another lawsuit has pre-empted the claimed “vindication” of the Pope study. A federal appellate court overturned the EPA regulations in May 1999 on a number of grounds. The case is pending before the Supreme Court.

No doubt more research will continue into the potential health effects of air pollution. But another byproduct of the Pope study controversy may change the shape of the next scientific debate. Because of difficulty in obtaining the Pope study data, a federal law was enacted in October 1998 requiring that federally funded scientific data used to support federal policy must be publicly available through the Freedom of Information Act.

With any luck, the next debate over the potential health effects of air pollution won’t be hampered by EPA’s “secret science.”

Steven Milloy is a biostatistician, lawyer, adjunct scholar at the Cato Institute and publisher of Junkscience.com.

Saving Secret Science

By Steven Milloy and Michael Gough
May 24, 1999, New York Post

Rep. James Walsh, an upstate Republican, apparently wants New Yorkers to spend billions of dollars annually based on the Environmental Protection Agency’s “secret science.”

In 1996, the EPA proposed new, more stringent national air-quality standards, estimated to cost New Yorkers as much as $3.3 billion annually and a loss of 10,000 jobs.

The EPA’s justification? It claimed the rules would save 15,000 American lives per year.

But that claim was based on a single scientific study (the “Pope study”). And that study wasn’t published in (and therefore peer-reviewed via) a top-line journal such as the New England Journal of Medicine, but a journal of the American Lung Association – a group that gets millions in EPA grants and lobbied actively for the new air-quality regulations.

Only the EPA has ever seen the Pope study data – and when Congress requested the data for independent examination, the EPA refused. Under political pressure, it relented – and the researchers took up the stonewall.

They claimed proprietary right to the data – event though the study was paid for with taxpayer dollars, was being used to impose huge costs and was requested by a Congress hardly interested in going into the scientific research business.

The EPA then cavalierly disregarded all objections – from its own science advisers, Congress and a bipartisan group of governors and mayors – and imposed the new regulations.

Industry sued. Congress legislated.

A new law extends the Freedom of Information Act to cover taxpayer-funded studies which are used to justify federal regulations. This would prevent agencies from regulating on the basis of “secret science.”

Now Rep. Walsh is leading a charge to have this “sunshine in the government” law repealed: He’s set to present an amendment today to block the law.

The EPA is confident the “data access” law is doomed: It’s now using the Pope study to justify new rules covering gasoline’s sulfur content and sport-utility-vehicle tailpipe emissions.

The EPA claims the rules will cost $3.5 billion a year – to save up to 2,400 lives per year. Is that claim accurate? Who knows? The EPA continues to refuse access to the Pope study.

Fortunately for now, a federal court this month found the EPA’s air-quality standards to be unconstitutional. But the EPA is unlikely to give up. Assuming the agency can figure out a constitutional way of regulating, Rep. Walsh is helping ensure that the science won’t face public scrutiny.

Steven Milloy is an adjunct scholar with the Cato Institute. Michael Gough is a senior scientist with the Competitive Enterprise Institute.