Instant relevance for policy research!

By | December 14, 2013

I’ve been working on a very interesting new post, but it’s not quite ready yet. In the meantime, I can’t resist sharing with you a rather choice item that I came across a day or two ago, as a result of googling the topic “relevance of public policy research” in an attempt to find at least one or two studies that I might offer to one of my students as examples of something useful.

As you may know, for the last 35 or so years I have practiced and taught about the fine art of behavioral science research. I’ve done quite a lot of it, much of it in the company of researchers much more talented and skilled than I; nonetheless, I think I’ve managed to train quite a few good researchers myself. Over the years, I’ve come to pretty much take for granted most of the basic rules that we believe we need to follow if we’re going to justify the degree of hubris necessary to generalize our findings – things like variables that vary, contexts that are reasonably representative, analytical tests that are appropriate to the data, etc.
You get the idea. Above all, I have come to have a deep and abiding respect for my data, and a confidence that if I treat my data respectfully, they will in turn yield up to me a measure of truth.

Well, apparently I’ve been doing it all wrong. At least according to Harold Goldstein of the Center for Public Health Advocacy in Davis CA. In a brief article in the Journal of Public Health Policy back in 2009 entitled, “Translating Research into Public Policy”, Mr. Goldstein observed that:

“Influencing and informing public policy are explicit goals of active living research. Active living researchers, motivated by a desire to improve health, often wonder why their findings do not translate more readily into changes in the “real world” of local, state, and federal policies.”

Well, he’s certainly right about that. In our heart of hearts, most of us researchers feel generally unappreciated, and cherish the belief that if only policymakers would really deeply read our studies and take to heart our recommendations, the world would be a far better place. The problem, of course, is that the world is an almost infinitely complicated place, while the world of research is an abstraction, simplified often to the point of unrecognizability. That’s pretty much how it has to be; even the largest and most comprehensive studies can contain only a limited number of the possible variables that might be included and test only small subsets of the possible relationships that might exist among them. We justify this course by saying that the variables are the really important ones and the relationships are the really significant ones to be tested. And often we’re right.

But then our findings, interesting and significant as they may be, get dumped back into the real world, and generally find themselves rapidly surrounding and losing their identity in a sea of other variables and relationships that weren’t part of our study but turn out to be the ones that policymakers really worry about. So with some sadness, we file our reports on the shelf, reach for the next RFP, and duly begin work on the proposal. And so it goes.

But according to Mr. Goldstein, the problem is that policy advocates get impatient when “…researchers want to maintain their objectivity and describe their research findings and limitations more thoroughly.” Oooh – that pesky objectivity! His recommendation:

“If, as an active living researcher, you want your work to have a more direct impact on the policy making process, I suggest you partner with advocates working on your issues before you start your next research project. Work together to decide what research is most needed, when the findings should be released, how the findings will be disseminated to have the most political impact, and what your respective roles will be in the policy development process.”

Now why didn’t I think of that! It’s so much simpler if we only decided in advance what our findings were going to be, defined mostly by what our sponsors would like them to be. It’s amazing how many resources could be saved by that little trick – one would scarcely need to go in the field at all. Just stay home and write up your report – so much easier!

Well, silly me. Here’s the solution to increasing the relevance of public policy research – just say what the politicians want you to say. Amazing how relevant that would make you. After all, we have Mr, Goldstein’s word for it.

Some days Google definitely delivers much more information than one is interested in knowing…