Dan Murphy: Survey Says … Wrong!

Recent surveys about why people have stopped eating meat seem to document a dramatic shift in popular opinion toward animal foods. But upon closer examination … no, they really don’t.

( FJ )

Recent surveys about why people have stopped eating meat seem to document a dramatic shift in popular opinion toward animal foods. But upon closer examination … no, they really don’t.

One of the few Math classes I ever truly enjoyed was a course in Statistics I was required to complete as an undergrad at the University of Oregon.

That was partly due to the celebrity status of the professor, an expert in test construction who had proven several times in the past that she could pass multiple choice exams written in a foreign language she didn’t speak — just because she knew all the tricks and tactics test makers use in writing questions.

But more relevant was the unit we studied on how to construct “neutral” survey questions. This was part of a certification in Community Health, a field that regularly uses surveys to assess public health priorities.

What we learned way back when, and which is even more relevant today, is the distinction between surveys that offer choices — essentially a checklist from which respondents can choose the answers that best express their views — and so-called “unprompted” surveys that use open-ended questions requiring people to identify what marketers call their “top of mind” responses.

With checklists, respondents are simply guided toward the answers the sponsoring org wants to document. It provides data that, at best, is unreliable and useful only as confirmation of what the pollsters already suspected.

But if you ask people, “What are your main concerns about ___________ [fill in the blank with whatever topic is being surveyed]?” then the replies are a far more accurate ranking of what really matters (and what doesn’t matter) to the population being surveyed.

Activists vs. Veggies

With that in mind, a recent article published in Mother Jones magazine titled, “We Asked Why You Stopped Eating Meat. Here Were Your Top 5 Reasons” offered a ranking of concerns a six-year old who’s watched a couple YouTube videos on eating meat could have accurately identified.

In order.

The only interesting part of MJ’s article, other than the occasional flashes of utter ignorance and hypocrisy displayed by the most ardent of the vegetarians they quoted, is the reversal of what activists continue to claim, and what veggies actually say about why they stopped eating meat.

At this point, I need to offer my mandatory disclaimer: I support anyone who chooses a vegetarian diet and I respect their choice as wholly legitimate. Period. End of disclaimer.

What’s objectionable isn’t the choices people make about what to eat, it’s the rationales offered up by veganistas who not only insist on a non-meat diet for themselves, but who demand that everyone follow their prescription, and worse, who obfuscate with equal ardor when it comes to honestly discussing the consequences of their wet dreams about Planet Vegan.

To convince the rest of the word, activists of late have focused laser-like on the (alleged) impact of livestock production on climate change, arguing that if only we all stopped eating meat and dairy foods we’d qualify as planetary saviors, as if that’s the most important action people could take to reduce greenhouse gas emissions. Given the scope and scale of media coverage, one would assume that environmental concerns would thus be at the top of the list of reasons why people say they quit eating meat.

But no. The No. 1 reason remained “Animal welfare,” with 354 of 493 respondents choosing that answer; only 199 people selected “Concern for the environment.”

And the other activists’ favorite whipping boy, the “horrors” of factory farming, garnered only 50 votes, finishing way behind “Health” (141 votes) and barely ahead of “Taste” (48 votes).

By the way, it’s easy to tell that the Mother Jones survey consisted of prompted responses for people to pick. Not only were multiple answers allowed — meaning that people could scroll down the checklist and mark as many choices as they liked — but when one choice was labeled “Distrust of Factory Farms,” it’s pretty obvious that a well-meaning editor composed the questions.

Nobody, unprompted, says “I really distrust those factory farms.” I hate them; I think they’re terrible; I want them banned. Those would all be plausible replies if someone identified factory farming as the reason he or she decided to go vegetarian.

Distrust? That’s what people feel toward banks, cable companies and used car salespeople. But nobody stops using ATMs, watching TV or driving cars as a result.

Finally, in assessing the validity of this and similar surveys purporting to confirm that America is embracing the Full Veggie, it is important to note one other factor relevant to constructing and conducting surveys: sample bias.

To arrive at data that’s credible, the population responding to a survey cannot be self-selected; it must be balanced in terms of age, gender, and other demographics.

That’s why if you’re one of those dinosaurs (like me) who still maintains a landline phone in your house, you get badgered relentlessly during election season. To compile voter projections that are reliable, pollsters have to keep calling people until they connect with a true cross-section of eligible voters in given jurisdiction.

In this case, the readers of Mother Jones are reliably predisposed to endorse vegetarianism and to embrace the allegations that raising livestock and eating meat have suddenly in this 21st century become the scourge of Mother Earth.

Thus, in no way does their half-baked survey represent an accurate barometer of public opinion, nor a dataset that reflects the reality of Americans’ dietary preferences.

My former Statistics professor could have figured that out even if Mother Jones had published the survey results in Swahili.

The opinions in this commentary are those of Dan Murphy, a veteran journalist and commentator.

Related stories: