Damned lies and statistics

Something that happened at work today reminded me of the college journalism class where my professor was warning us about the pitfalls that come along with reporting statistics. She told us about how, a few years back, UNC Chapel Hill did a survey of the salaries earned by alumni in different majors a certain number of years after graduating. And guess what the top-earning major was? Geography.

We digested that for a moment, and then somebody said that they had no idea that geology majors earned so much later in life. They don’t, the professor said. But one UNC geography major did quite well for himself, and his name was Michael Jordan.

She told us this in order to drive home how meaningless stats such as “average” can be if they’re not reported correctly. I think all of us are aware of how statistics can be manipulated to show one thing or its exact opposite.

For example: the college where I work was founded by members of a religious group – let’s call them… Oh, hell, I can never think of anything in these situations. How about The Ramones? Sure. Okay, so we’re no longer governed by the Ramones, but Ramone values are still a large part of our identity. Today I asked the religious studies coordinator, a Ramone himself, how many self-identified Ramones are in the student body this year.

So, last year, nine percent of the student body were Ramones. This year it’s eight percent. BUT, the actual number of Ramones has gone up. The student body is also larger, so the Ramone percentage of the total is smaller… even though there are more of them – 25 percent more, actually. See how fun this is? Have a headache yet?

(For the record – last year there were 27 Ramones, and this year 36, with 200ish more total students.)

I think of this every time I see another college report that its “average” SAT score is up, or that its student body has grown X-percent. I think of it when I see a corporation brag that its profits have gone up X-percent. It *really* chaps my ass when I see sloppy stats applied to sports, removed from all context or common sense.

It happens with political reporting all the time. Take, for instance, a survey asking random people if Sarah Palin’s resignation affected one’s view of her. Well, if I’m asked that question and I answer honestly – Nope, not one bit. But I already didn’t like her. That survey doesn’t really reveal what it claims to reveal.

The next time you see a stat, demand the whole story: sample size, wording of questions, all of it.