Herding Cats: Study Results

More study is needed. That's how we ended reports at the research center I worked at in the 1980s. Maybe this is not inevitable in other fields, but educational research just gave rise to a lot more questions. Especially if it led to conclusions you didn't want to find, that is, the funding agency didn't want to find. All research is paid for by someone. Except my UU cat poll, whose results are posted in the upper right. This poll was undertaken with no funding at all. In that sense, and only that sense, it is free of bias, unlike drug studies, which someone (probably a drug company) has to pay for, after all. And that funder usually wants certain results.

Other than that, how should we evaluate this independent poll?

Well, the n, or number of respondents, is small. This is not that unusual. I've seen studies of whether mental illness can be cured that made important, sweeping conclusions on the basis of studying 35 people, people who were hospitalized for mental illness.

Then there's self-selection bias. This is always big when you just poll whoever comes in the door and feels like taking part in a poll. They have selected themselves to be respondents, first by just coming in the door - and my blog is more likely to attract cat people than dog people. Second, they have agreed to be the people who respond. Why? You never know.

The qualification of respondents kind of slipped. We don't have any concrete evidence to begin with that they were Unitarians, the subject under study. They implied they were by taking the poll. But nobody checked their wallet to see if they were carrying a copy of the Seven Principles. They could just be cat lovers who will do anything to forward the cause. (But we didn't check them for cat hair, either.)

There's the design of the questionnaire itself. This one was designed in haste by a panel of one, no meetings involved, and had to adhere to the rigid grid available. Did it really logically lead to any conclusion about Unitarians being highly individualistic? Maybe we like cats because we're well-behaved people and they act out for us.

Now. We have just skimmed over the first part of every research report, except that most will discusse issues of statistical significance that it is possible no one really understands. This part is called The Limitations of the Study, and you have to get it out of the way. I suppose that was the most valuable thing I learned working there - every study has its limitations, just as every novel has a hole in the plot you could drive a truck through. The other thing I learned was that that needn't stop you from proclaiming you learned something valuable. It was usually what the study "suggested."

This study may suggest (notice the cautious language) that Unitarian Universalists are more likely to own cats than dogs. From there, to the question of what that could mean is a huge leap. But it validates my intuition to my personal satisfaction, and I'm the one writing the report. That's the way research goes.

Furthermore, certain questions point the way for future research, like, What is that one "other" pet somebody owns? An iguana? A hissing cockroach? One of those tiny cute horses? And is it possible that all Unitarians own a pet, or to put it another way, that there is no Unitarian who does not own a pet? I don't think so. Certain friends come to mind. They know who they are.

credits:
The photo is one of many taken at a special cat shelter. You should be able to get to the website by clicking on the title of this post.

And thanks to all who participated. I'm sorry I told you you could vote as many times as you wanted. I have been informed that isn't true. I hope the same holds next Tuesday, as well, because they only let me vote once.