The EPA's Secret Science

By Steven Milloy
February 2, 2001,

“Independent Institute Finds Key Pollution Studies Are Sound Science,” blared a recent press release from the American Lung Association.

Yet one of the most controversial “junk science” issues of recent years — whether air pollution causes premature death — will remain controversial for the foreseeable future despite activist claims and media headlines.

The ALA goes on to say studies show “once again that air pollution shortens lives, and strong [Environmental Protection Agency] standards are needed to protect public health.”

But the reality is somewhat different — at least according to Health Effects Institute, the organization whose research the ALA cited.

The saga started in November 1996 when the EPA proposed more stringent regulation of airborne particulate matter, otherwise known as soot. The EPA initially claimed further regulation would prevent 20,000 “premature deaths” every year at an annual cost of $8 billion. By valuing each life saved at about $5 million, the EPA estimated the monetary benefits of saving these lives at $100 billion per year.

But the validity of the EPA’s estimates is in question. In particular, the estimate of premature deaths was based on the “Pope study,” so called for its lead researcher, C. Arden Pope of Brigham Young University. The Pope study reported that airborne particulates were associated with a 17 percent increase in premature deaths. But this result constituted only a weak statistical correlation — not scientific proof of a cause-and-effect relationship between air particulate pollution and premature death.

Although the study included more than 550,000 people, the researchers did not measure the level of exposure to air pollution for even one study subject. Instead, they guessed how much pollution these individuals might have encountered. The researchers also failed to look at the subjects’ diets, income, health histories, genetic predispositions to illness, exercise habits and social habits — all well-established risk factors for premature death.

The study did adjust for some factors — including smoking habits, education level and occupational exposures — but additional adjustments could easily negate the purported 17 percent increase in risk.

A further problem was that no one has ever demonstrated how typical levels of airborne particulates could cause premature death.

In this context, the reported correlation could easily have been a statistical artifact. According to the National Cancer Institute, “In epidemiologic research, [risks of less than 100 percent] are considered small and usually difficult to interpret. Such increases may be due to chance, statistical bias or effects of confounding factors that are sometimes not evident.”

But the EPA did not back down — even when an elementary statistical error in the EPA’s calculations was discovered that knocked down the lives-saved estimated to 15,000 or when some economists estimated the proposal’s cost to exceed $100 billion annually.

The controversy prompted Congress to ask the EPA to produce the Pope study’s raw data so independent scientists could check the results.

The EPA initially balked, saying there was no purpose in any re-analysis of the EPA-funded study. Finally, after issuing the regulations, the EPA did provide access to the data — to the Health Effects Institute, a Massachusetts-based research organization funded by the EPA and the auto industry.

HEI issued its report last week, three years after the regulations were finalized. The results are not quite as the ALA press release touts.

Using essentially the identical methodology as the Pope study researchers, HEI produced virtually the same results — hardly a surprise. A naive media reported this “replication” in headlines such as “Research on Air Particles Passes Muster” and “Studies Back Particulates’ Link to Death.”

But the re-analysis has essentially the same shortcomings as the original study. The researchers did not factor into their analysis the effect of diet or genetics on death rates. Data on exercise habits were factored in, and ended up reducing risk estimates by almost 30 percent. But the quality of the data on exercise is debatable; incredibly, it indicates no difference in death rates between non-exercising study subjects and heavy-exercising study subjects. Better quality exercise data could have an even more dramatic effect on the numbers.

There is a similar data-quality problem with health history. The re-analysis paradoxically indicates that healthy study subjects had twice the risk of premature death of diseased study subjects.

Neither of these data problems is surprising given how the study data were collected. The American Cancer Society amassed the data by having 70,000 volunteers — not trained data-collection specialists — go to their friends, family and neighbors and ask personal, health-related questions. No effort was made to verify or validate the study subjects’ responses.

It’s no wonder the HEI concluded its report: “It is important to bear in mind that the results of our re-analysis alone are insufficient to identify a causal relation with mortality.”

So why did the ALA jump to its conclusion? That question perplexed HEI’s president Daniel Greenbaum, who noted the ALA issued a press release last April titled “New Health Research ‘Vindicates’ EPA; Soot Particles Are Deadly, Lung Association Notes” — three months before the HEI report was made available to the public.

The explanation, though, lies with the finances and politics of the ALA. As first reported by Investors Business Daily in January 1997, “The ALA has had a long — and lucrative — relationship with the EPA.” In the years before the EPA air-pollution proposal, the agency gave the ALA almost $5 million — despite the ALA suing the EPA almost every year claiming the agency wasn’t complying with the nation’s air-pollution laws.

“If you think the EPA is upset with the ALA suing them, think again,” said Scott Segal, a Washington, D.C.-based attorney. “Truth be known, the EPA wants to be sued, because every time they are sued it expands the reach of the Clean Air Act.”

Unfortunately for the ALA and the EPA, another lawsuit has pre-empted the claimed “vindication” of the Pope study. A federal appellate court overturned the EPA regulations in May 1999 on a number of grounds. The case is pending before the Supreme Court.

No doubt more research will continue into the potential health effects of air pollution. But another byproduct of the Pope study controversy may change the shape of the next scientific debate. Because of difficulty in obtaining the Pope study data, a federal law was enacted in October 1998 requiring that federally funded scientific data used to support federal policy must be publicly available through the Freedom of Information Act.

With any luck, the next debate over the potential health effects of air pollution won’t be hampered by EPA’s “secret science.”

Steven Milloy is a biostatistician, lawyer, adjunct scholar at the Cato Institute and publisher of

Disinfecting the anti-bacteria debate

By Steven Milloy
July 27, 2000, Washington Times

Are we too clean? Should we make homes safer for germs? That is the message you’ll get from today’s National Health Council conference, “Antibiotic Resistance: A Serious Public Health Threat.” But there is more to this message than its messenger will disclose. Continue reading Disinfecting the anti-bacteria debate

A Scoop of Debunkey Monkey, Please

By Steven Milloy
July 8, 2000,

The Environmental Protection Agency says the much-ballyhooed environmental contaminant dioxin is 10 times more dangerous than previously thought. Environmental activists already have begun a food scare campaign centered on dioxin. But amidst this eco-terrorism, the long-running and costly health scare over the dreaded dioxin is finally debunked — courtesy of ice cream maker, Ben & Jerry’s. Continue reading A Scoop of Debunkey Monkey, Please

Vinyl IV Bags: Media lose message

By Steven J. Milloy
March 27, 2000, Chicago Sun-Times

Deerfield-based Baxter Healthcare has been vindicated on a major health scare. But don’t expect to read about it in the media. It’s more fun to scare readers about vinyl IV bags causing cancer than it is to set the record straight. Continue reading Vinyl IV Bags: Media lose message

Fear of margarine: The trans fat myth

By Steve Milloy
November 29, 1999,
(Note: The many links in this article are no longer active but have been left in for historical purposes.)

The Food and Drug Administration recently proposed (FDA proposal home page | Federal Register notice) to amend its regulations on nutrition labeling to require the amount of trans fatty acids in foods be included in Nutrition Facts panels. But the science behind this move is suspect.

A recent editorial in the New England Journal of Medicine reviewed the epidemiolgic evidence linking trans fatty acids to heart disease. The editorial concludes “Metabolic and epidemiologic studies indicate and adverse effect of trans fatty acids on the risk of coronary heart disease.” Below (in italics) is what the editorial says about the epidemiology studies — the studies that should validate the theories developed from the metabolic studies. My comments are in bold. Decide for yourself whether trans fats are guilty as charged.

  • In a case-control study of subjects in the Boston area, we found a strong and significant positive association between the intake of trans fatty acids, assessed with the use of dietary questionnaires, and the risk of acute myocardial infarction. The relative risk of acute myocardial infarction for the quintile with the highest intake of trans fatty acids as compared with the quintile with the lowest intake was 2.4 (P for trend less than 0.001); this association was entirely explained by the intake of these fats from hydrogenated vegetable oil.

    This small study (only 239 patients from the Boston area) did not consider non-dietary risk factors for heart disease other than age and sex. Astonishingly, smoking, exercise level, health history, family health history and alcohol consumption were not considered as confounding risk factors.

  • Bolton-Smith et al. performed a cross-sectional analysis of the association between the intake of trans fatty acids and the presence of previously undiagnosed coronary heart disease among participants in the Scottish Heart Study. The intake of trans fatty acids was positively correlated with the ratio of LDL plus very-low-density lipoprotein cholesterol to HDL cholesterol. The odds ratios for coronary heart disease in the quintile with the highest intake as compared with the quintile with the lowest intake were elevated but not significantly so (1.26 in women and 1.08 in men).

    The results from this study were weak statistical associations that were not statistically significant — meaning the probability was unacceptably high the barely detectable associations could have occurred by chance. Even the study authors conclude,”The results, therefore, do not support a major effect of dietary trans fatty acid…”

  • Studies in which the composition of fatty acids in tissue or plasma was used as a marker of trans-fatty-acid intake have yielded conflicting results. With one exception, however, these studies have been too small to detect an association reliably. The results of the only large study, which included 671 men with acute myocardial infarction from eight European countries, were inconclusive. The overall analyses revealed no association between the intake of trans fatty acids and the risk of myocardial infarction.

    No comment necessary.

  • The relative risk of coronary heart disease associated with an absolute increase of 2 percent in the intake of trans fatty acids was 1.36 (95 percent confidence interval, 1.03 to 1.81) in the Health Professionals Follow-up Study

    The result spotlighted is so misleading as to constitute scientific misconduct. The relative risk of 1.36 is a raw result, without any adjustment for other heart disease risk factors. When other risk factors are adjusted for — including age, body mass index, smoking habits, alcohol consumption, physical activity, history of hypertension or high blood cholesterol, family history of myocardial infarction before age 60, profession, and fibre intake — the weak relative risk is substantially reduced (by more than 50 percent) and becomes statistically insignificant.

  • The relative risk of coronary heart disease associated with an absolute increase of 2 percent in the intake of trans fatty acids was … 1.14 (95 percent confidence interval, 0.96 to 1.35) in the Alpha-Tocopherol Beta-Carotene Cancer Prevention Study

    The spotlighted result is a weak association that is not statistically significant. This study consisted of 21,930 male smokers. Can you really study dietary factors for heart disease in a population where the basic lifestyle (i.e., smoking and its attendant unhealthy tendencies) is a risk factor for heart disease?

  • The relative risk of coronary heart disease associated with an absolute increase of 2 percent in the intake of trans fatty acids was… 1.93 (95 percent confidence interval, 1.43 to 2.61) in the Nurses’ Health Study.

    This is my favorite study of the bunch. Check out its results.

    • It reports no statistically significant association between total fat intake and risk of cornary heart disease.
    • It reports no statistically significant association between animal fat intake and risk of cornary heart disease.
    • It reports no statistically significant association between saturated fat intake and risk of cornary heart disease.
    • It reports no statistically significant association between cholesterol intake and risk of cornary heart disease.
    • The reported association between trans fat and coronary heart disease is only statistically significant for the highest consumption of trans fats — but it’s still a very weak statistical association (relative risk = 1.53).

    This study basically reports that all we’ve been told about the association between fat consumption and heart disease is not supported by data collected from 90,000 nurses over a period of 20 years. So either the study data is wrong or the public health establishment has been wrong about fat consumption being associated with heart disease risk. If the study data is wrong, then I doubt the trans fat result. If the public health establishment is wrong then why should we believe it about trans fat when it has been generally wrong about fat consumption for the last two or three decades?

That’s the epidemiology supposedly supporting the proposition that trans fats are so much of a risk for heart disease they need to be labelled. Are you convinced yet?

But there’s more. Check out the authors of the studies discussed above. See if you notice anything unusual (like the underlined names).

  • Ascherio A, Hennekens CH, Buring JE, Master C, Stampfer MJ, Willett WC. Trans-fatty acids intake and risk of myocardial infarction. Circulation 1994;89:94-101.
  • Bolton-Smith C, Woodward M, Fenton S, Brown CA. Does dietary trans fatty acid intake relate to the prevalence of coronary heart disease in Scotland? Eur Heart J 1996;17:837-45.
  • Aro A, Kardinaal AF, Salminen I, et al. Adipose tissue isomeric trans fatty acids and risk of myocardial infarction in nine countries: the EURAMIC study. Lancet 1995;345:273-8.
  • Ascherio A, Rimm EB, Giovannucci EL, Spiegelman D, Stampfer M, Willett WC. Dietary fat and risk of coronary heart disease in men: cohort follow up study in the United States. BMJ 1996;313:84-90.
  • Pietinen P, Ascherio A, Korhonen P, Hartman AM, Willett WC, Albanes D, Virtamo J. Intake of fatty acids and risk of coronary heart disease in a cohort of Finnish men: the Alpha-Tocopherol, Beta-Carotene Cancer Prevention Study. Am J Epidemiol 1997;145:876-87.
  • Hu FB, Stampfer MJ, Manson JE, Rimm E, Colditz GA, Rosner BA, Hennekens CH, Willett WC. Dietary fat intake and the risk of coronary heart disease in women. N Engl J Med 1997;337:1491-9.

ALL THE STUDIES supposedly showing trans fats are associated with heart disease risk involved Alberto Ascherio and Walter Willett.

You may also be interested in knowing who authored the editorial: Alberto Ascherio, Martijn B. Katan, Peter L. Zock, Meir J. Stampfer,and Walter C. Willett.

Should the FDA be taking action because of a duet of scientists whose results are so thin? Has the FDA ever heard of the scientific method and its requirement for independent replication of scientific results?

PSA test: Promoting Stress and Anxiety?

by Steve Milloy
October 6, 1999,

I hate to admit it. But I’m at that age where prostate cancer enters the mind – especially since my father just completed a course of radiation treatment for his prostate cancer.

Prostate cancer is the most commonly diagnosed cancer among U.S. men. And at about 40,000 annual deaths, it is the second deadliest behind lung cancer. Prostate cancer is the male version of breast cancer. It’s not preventable and early detection is the key to survival. But – and it’s a big “but” – controversy has arisen over early detection.

The prostate is a chestnut-sized gland in the make pelvic area. It plays a role in generating seminal fluid that helps transport sperm. For unknown reasons, the gland can enlarge and cancer can develop. Some cancers are fatal. Some aren’t. While most prostate cancer occurs in older men, it can strike younger men as well. Half of prostate cancer deaths occur in men over the age of 75. Fewer than 5 percent of deaths occur in men who were under 60 at diagnosis.

Until about 10 years ago, prostate cancer was detected by a technique euphemistically named the “digital rectal exam.” But rectal exams often don’t catch cancers until after they have spread – at which point it may be too late to cure. Then came the PSA test, a blood test that measures the level of prostate specific antigen. PSA is usually elevated in men with prostate cancer, but can also be elevated, usually to a lesser degree, in men with benign prostate growth.

The PSA test did wonders for diagnosing prostate cancer. The number of men diagnosed with prostate cancer in 1985 was about 85,000. By 1996, the number diagnosed had jumped to almost 320,000. Sounds great right? It was early detection we needed and early detection we got. But maybe not.

Though more prostate cancer is being detected now than 10 years ago, it’s not that more men are getting prostate cancer. It’s just that more prostate cancer is being detected by the PSA test. Not all prostate cancer is fatal. In older men, prostate cancer can be slow-growing so that they will die with it, not of it.

An elevated PSA test does not automatically mean that cancer is present. Like the digital rectal exam, a PSA a test can erroneously indicate prostate cancer and erroneously fail to detect a prostate cancer. So while the screening may lead to life-extending diagnosis and treatment, it can also lead to treatment of cancers that don’t need to be treated – or even cancer that a man may choose not to treat. Treatment may cause urinary incontinence or impotence.

Several years ago, controversy arose over whether routine screening for prostate cancer.

The American College of Physicians says that routine prostate-cancer screening “is not for everyone” and that it is “a complex decision that patients should make after talking to their physician, understanding the risks and benefits, and coming to an informed, individualized decision. A urologist said in the British Medical Journal that [PSA testing] “merely Promotes Stress and Anxiety.”

But the American Urology Association and the American Cancer Society recommend that, annual PSA testing be done annually beginning at age 50, and earlier for men at high risk. The developer of the PSA test, Dr. William J. Catalona, professor and chief of urology and the Washington University School of Medicine says “PSA testing has exhibited all the features of an effective cancer-screening tool.” But note that he didn’t say it saves lives. Why? Because there’s no proof it has.

The National Cancer Institute criticized the American Cancer Society in 1992 for acting hastily in recommending routine screening without waiting for proof that mass screening reduces mortality. The ACS’ chief medial officer offered the weak defense that “Physicians and the public are demanding to be told what to do.”

Until recently, there have been no controlled studies on whether PSA testing reduces the prostate cancer death rate. The first clinical trial to examine the value of PSA screening, released in May 1998, reported that PSA screening led to a 69 percent reduction in prostate cancer death rates. While sounding impressive, researchers immediately raised questions about potential bias in the study data. Men who agreed to be screened may have been healther than those who did not agree.

Now, a new study in the Oct. 6 Journal of the National Cancer Institute reports that PSA levels may mean something entirely different than originally thought.

So what do you do? How do you choose between the American College of Physicians and the American Urological Association? Does it help to know that the ACP recommendation is “evidenced-based”? The ACP examined and reviewed more than 40 studies to determine the potential benefits of screening, the complication rate of treatment, and the balance of benefits harms and costs. In contrast, the AUA recommendation represents a consensus of expert opinion. Evidence vs. expert opinion? Wouldn’t you think expert opinion would be based on evidence? Confused? As my father-in-law, a prominent south Florida urologist said, “Urologists have the most experience, but they also have the most to gain from routine screening.”

What do urologists have to gain? You decide. PSA testing costs about $100. An elevated PSA could lead to more testing or a biopsy costing around $1,500. The prostate testing market is worth about $200 million to $300 million annually. Until recently, the developer of – and the leading cheerleader for – the PSA test, William Catalona, was receiving $1 million per year “to study the effectiveness of the PSA test,” largely from Hybritech, a pharmaceutical firm that manufactures the PSA test.

Recently, Catalona criticized the American College of Physicians for recommending against routine screening in favor of physicians and patients discussing the potential benefits and known harms of screening diagnosis and treatment and making individual decisions. Catalona wrote “This sounds reasonable but is impractical because busy doctors lack time for long discussions.”

Prostate cancer can be life-threatening. If your physician doesn’t have the time to discuss your options, get a new doctor. And whatever you do, don’t fall for the unproven and undeciphered PSA test.

Steve Milloy is the publisher of