Dan Murphy: Even ‘Real’ Stats Can Be False

One of the more interesting classes I remember from way back as an undergraduate was a specialized statistics course designed for Health Education majors like myself.

The main reason I enjoyed the course was the “celebrity” professor who taught us. I say celebrity because she had been profiled on local television due to her claim — which she’d repeatedly backed up — that she could pass any multiple-choice test without preparation … even one that was written in a foreign language she didn’t speak!

As a result, the professor was a bit of a legend on campus, and without fail, one of her students would ask her during the first week of the term, “Hey, how is it possible to pass a test in a language you can’t read?”

It was the prefect softball question to tee up the study of statistics, and she was happy to explain about the science of predictive valuation and regression to the mean, supplemented by the art of test construction, which was where the prof got to strut her stuff about passing any multiple-choice test.

For example: If one of the answers to a question is significantly longer than the others, it’s highly likely that stem is the correct one, since teachers or test-makers want to make sure the correct answer is absolutely accurate.

Anyway, beyond learning how to scam your way through test-taking — a college student’s best friend — the course also drilled into us the importance of understanding basic statistics as they’re used in the media, by policymakers and in medical contexts.

Even percentages, which we all believe we fully understand, can be grossly misrepresented, often unintentionally.

Just the other day, for instance, a local commentator was lamenting the scale of the opioid crisis in King County (Seattle), noting the “nearly 50% increase in the death toll” over the last 10 years (2008-2017).

Of course, any drug-related death is tragic, and a 50% increase in fatalities does sound terrible. However, the statistics the commentator referenced included deaths from alcohol and recreational drugs, such as cocaine. Plus, a deeper dive revealed that in 2017 fatalities spiked unusually, and by merely substituting data from 2006 to 2015, the increase was less than 30%.

Moreover, if prescription opioids alone are disaggregated from the other causes of overdose, the increase in the death rate from 2008 to 2017 was … wait for it … zero.

And if we account for King County’s 12% population growth from 2008 to 2017, then the rate of deaths from opioids actually decreased by that same percentage over the last decade.

By the way? The death rate for all drug-related overdose fatalities in King County — at its highest level in 2017 — was 11.9 per 100,000 people. Know what the national death rate per 100,000 people from accidental poisoning is? It’s 14.8 and for the record, that’s 25% higher than drug and alcohol overdoses.

Dissecting the Data
All that is by way of responding to a recent boast, headlined in the UK media, that “53% of participants” in something called the Vegan Summer Pledge have (allegedly) chosen to continue avoiding animal foods following the end of the 30-day challenge that was promoted by various celebrities showcasing their own conversions to a life of eating only plant-based foods.

The event was sponsored by the British activist group Animal Aid, which was quick to proclaim that their post-program survey was evidence that the world is turning veggie.

However, as the example above shows, numbers aren’t always what they claim to be.

Here’s why: According to the group, a total of 3,764 people signed up for the challenge, receiving free recipes, shopping coupons and a connection to other participants on social media, which would likely skew the statistics about how many people publicly “pledged’ to go veggie.

But here’s the key stat: The 3,764 participants included 1,392 vegetarians and 651 pescatarians (who only eat fish along with their vegetables). Since 1,995 people in the survey claimed that they’re now vegetarians, that means that there were only 603 “new” converts, which undercuts the group’s headline proclaiming that, “53% of People are Staying Vegan” after the challenge.

The increase was only 25% (603 new converts out of 2,372 non-vegetarian participants).

And since it’s not exactly a life-changing conversion to go from pescatarian to vegetarian — how much fish can anyone eat on a daily basis? — then the Summer Challenge produced 1,995 vegans out of 2,042 non-meat eaters.

Not exactly the revolutionary increase that Animal Aid would like its followers to believe actually occurred.

And there’s one other statistical factor that must be considered, as well: sample selection. To obtain reliable data, a sample cannot be a group of self-selected people. When Animal Aid recruited its 3,764 participants, they didn’t do so by balancing age, gender, dietary preferences and other factors to come up with a sample that reflected the general population of Britain. They simply signed up thousands of people who were already interested in going vegan.

Based on that alone, even the inflated figure of 53% of participants going vegan isn’t all that impressive.

Which is a pretty fair characterization of the vegan diet, as well.

Editor’s Note: The opinions in this commentary are those of Dan Murphy, a veteran journalist and commentator.

Comments