Monday, March 23, 2009

A little exercise in reasoning and in politicized science reporting

The Washington Post had an article with this headline that caught my eye:
"Study Finds Eating Red Meat Contributes to Risk of Early Death."

Does the study really "find" that? Let's see what the article actually says...

The lead paragraph: "Eating red meat increases the chances of dying prematurely, according to a large federal study that offers powerful new evidence that a diet that regularly includes steaks, burgers and pork chops is hazardous to your health."

Oh my! Let us read on to see how this study demonstrates this. The next graph says:

"The study of more than 500,000 middle-age and elderly Americans found that those who consumed the equivalent of about a small hamburger every day were more than 30 percent more likely to die during the 10 years they were followed, mostly from heart disease and cancer. Sausage, cold cuts and other processed meats also increased the risk."

Hmmm...can you see the logical fallacy revealed here?

The second paragraph tells us the study discovered an association. People who eat certain meats are more likely to die. But is that the same thing as causation?

The answer, dear reader, is that it is not. Why not?

Well, for example--aren't you curious, as I am, to know anything else about these folks who died earlier? And about those who, despite stuffing all that meat down their gluttonous throats, did not proceed to die earlier? Why weren't they 100% more likely to die earlier?

Perhaps because of...other variables? Such as exercise, weight, smoking, stress, other vices...who knows what?

The article says, later, that the study accounted for those variables. Over a half-million volunteers filled out detailed questionnaires in 1995; then, "Over the next 10 years, 47,976 men and 23,276 women died."

Okay; that's concerning, no question. But again, this inquiring mind wants to know: how inquisitive were the researchers into other behaviors and stresses, and changes in habits, over the following ten years?

The article proceeds to say that "routine consumption of fish, chicken, turkey and other poultry decreased the risk of death by a small amount, the study found."

It occurs to me that they may have cause-and-effect backwards: perhaps people who lead healthier lifestyles tend to eat fish and turkey; and those attached to decadence prefer those "bad meats" we just heard about.

In the interest of full disclosure, I happen to love all those "bad meats" and I don't keep measures of how much I eat; and I am rather more, er, "ample" than I ought to be. Certainly, there may be a direct causation there.

Trouble is, there is also interesting evidence that folks who go on diets consisting overwhelmingly of such meat--bad plus good--lose weight! Hmm, how to factor that in?

Now, I would have ignored this article, as garden-variety sloppiness, until I read this:

"'This would be the Rolls Royce of studies on this topic,' said Barry M. Popkin, a professor of global nutrition at the University of North Carolina, who wrote an editorial accompanying the study. 'This is a slam-dunk to say that, "Yes, indeed, if people want to be healthy and live longer, consume less red and processed meat."'"

I suppose it's rather uppity of me to cross swords intellectually with Professor Popkin, but--wouldn't the "Rolls Royce" of a study such as this eliminate other variables, in order to demonstrate, clearly, cause-and-effect?

The Meat Institute ("boo! hiss! we know their agenda) makes the reasonable--but not devastating--observation that "the findings...were based on unreliable self-reporting by the study participants." Which--if such self-reporting is unreliable (I don't know if it is, but it seems that it might be), it might mean anything: the early-to-die folks might have eaten rather more meat than they wanted to report, or done other unhealthy things they didn't like to talk about, or overstated how much they exercise, etc. Or it might mean something else; or it might be that self-reporting is, in fact, as reliable as other tools. Might have been nice to have had a rejoinder to that point; but the Washington Post didn't expect anyone would take seriously what a trade group would say in, harumph!, obvious self-interest!

After that offensive interlude, the article takes us back to the voice of sweet, non-profit wisdom: the National Institutes of Health, AARP, and the Harvard School of Public Health! Surely we can trust them!

Now, sarcasm aside, I concede the study itself may well be more probative than is clear from the article; my criticism is directed against the article, and I am, yes, skeptical about just what the study does, and does not, demonstrate. We would all do well to be skeptical of such things, particularly as reported in the media.

But it was the final two paragraphs that caused me to chuckle knowingly, and write this post. Just tell me if you can see what it might have been:

In addition to the health benefits of reducing red meat consumption, a major reduction in meat consumption would probably have a host of other benefits to society: reducing water shortages and pollution, cutting energy consumption, and tamping down greenhouse gas emissions -- all of which are associated with large-scale livestock production.

"There's a big interplay between the global increase in animal food intake and the effects on climate change," Popkin said. "If we cut by a few ounces a day our red meat intake, we would have big impact on emissions and environmental degradation."

Nope, no political agenda there, is there?


mamacantrix said...

Yikes. Just that. Yikes. And, oh yeah, I'll have mine medium rare, please.

Anonymous said...

It's hard to get accurate information about health issues in this country.

Something I ran across that seemed interesting was, in the US there is an ongoing persecution against overweight people for "making themelves get diabetes."

Yet in the UK diabetes is considered a genetic disease. It's true that diabetic Brits can be fat too, just like here, but their medical experts point out that in diabetes one's insulin is unable to metabolise sugar into energy, as is normal, and does not know what to do with sugar molecules, therefore it transforms them into fat.

Hence we have US citizens supposedly making themselves diabetic by being fat, while at the same time we have our neighbors across the sea getting diabetes via heredity and as a result of the diabetes, getting fat.

Yet has any American been exposed to this concept? It's as you say re the red meat brouhaha, alot is left out in order to support a favored theory.

If the food Nazis really want to help, let them go after all the unhealthy chemicals, hormones, and pro-marketing additives that are shoved into red meat. Is that stuff healthy? I don't think so!


Greta said...

Question to always ask is who paid for the work to be done or had anything to do with it. The statement the devil is in the details always applies. Kind of like the study that showed according to the story the huge uptake in the non relgious.

My spouse comes away from TV with a lot of garbage provided those who are too lazy to ask questions.

Having seen your manner of questioning brings back the exchange on the sacrament of pennance. Makes me wonder about coming for one of your times for that sacrament. Are you equally inquistive about everything?

If so, bet you were a treat in the seminary...

Father Martin Fox said...


Oh, I handle confession very differently; I don't poke about in people's personal business the way I do a news story.

joewxman said...

i find these studies often times ridiculous. 75000 people of the 500,000 followed died which was a 15% death rate. What is "middle age"? What is "elderly"? How many of these people were about to die from old age? Of course its all related to the Goreacle and global warming since if we eat less meat then there will be less cows farting all over creation.

Peyton's said...

Remember when Regan was in office? Yeah, me too, apparently according to my dad, he was personally responsible for us having too much homework, and still expresses to so in conversation today.
I don't get why people hang onto falsified information, it is as though if they all admit that global warming isn't in existence then they all would be humiliated and they cannot handle that. So even though,(since Obama was elected) scientist now say that the earth is actually cooling, and global warming is not a factor. Those that cannot let it go, now say it is stalled, and will continue to warm in a few years. It isn't as though people can really control the Earth or Mother Nature, any more than people can control life and death, although they sure want to try.

Tom said...

If 14% of a population dies, and meat eaters are 30% more likely to die than non-meat eaters, then meat eaters die at a rate of between 14% and 18.2%, while non-meat eaters die at a rate of between 10.8% and 14%. (The exact values depend on the ratio of meat eaters to total population.)

A difference of 4.2% in the rate of death is meaningful -- in each group of twenty-five, you'd expect one more death than you would otherwise -- but maybe not front page, above the fold news.

Frankly, I'm more concerned that men appear to be 206% more likely to die than women, though I suppose if we got rid of all the men it would have big impact on emissions and environmental degradation.

Anna said...

Sounds very similar to the study that was done on women. Proving that even 1 alcoholic beverage a day increased their chances of breast cancer.

BUT, what was not looked at is whether there was an increase in total deaths or a decrease due to decreased heart disease.

Anonymous said...

The accuracy of self-reports can vary a lot. The most accurate probably would be twittering people frequently but at various times on various days for at least a decade - what was everything you ate the last thing you ate?

Asking them one time what their average diet was over the last number of years (which seems consistent with the WP summary) - not good. We know people are bad at remembering what and how much they eat, because if you have them keep a food diary and then isolate them and feed them exactly what was in the diary they lose weight.

The problem with doing research on people, is that very rarely can you control any of the interesting variables. Even if you randomly assign people to eat and exercise certain ways, and you can prove they comply, there's their earlier life to consider. (What your grandmother ate while pregnant with your mother has been shown to affect your health)

You can gather information about everything you think might be relevant and try to statistically control for it, but it's still associations rather than cause and effect and variables often cluster. (I saw an article recently where they couldn't look at prenatal vs. postnatal smoking because there weren't enough people who only did one.)

My sister received her journalism degree from Marquette and I believe she was required to take statistics, but I don't think most journalism schools have that requirement.