Effective research education

By | November 12, 2015

There’s more LinkedIn debate about the best way to teach research tools. As usual, I have weighed in late in the discussion, but it’s something that over the years I’ve geven a lot of thought to, so I thought it might be worth summarizing my observations here. As I wrote there:

“It’s a bit presumptuous of me to jump in at this point after so many excellent points of been made, but a couple of different observations do come to mind. Jimmie observed earlier the difficulties posed by a “one-size-fits-all” approach. In fact, it might be helpful to think of this problem as involving a hierarchy of skills, not all of which need to be equally located in the same analyst. Most significant behavioral science research these days is conducted in teams of varying sizes, ranging from a couple of principal researchers supported by a few graduate students to large research groups with many analysts and highly specialized support. So when we are preparing students to enter the world of research, we ought to be preparing them with the same kind of team orientation and team skills that are increasingly prized in other parts of the productive world these days. In fact, group-based training – far from undermining skills development – might actually contribute to much more effective learning, since it would be within the professional context the students aim to enter.

The most effective way to learn how to do research is by actually doing research in collaboration with people who know what they’re doing. There is a substantial portion of “craft knowledge” that goes along with and enriches classroom and book learning, provides context and opportunity for learning about all the subtle distinctions that are made in real data analysis, and in general distinguishes the competent data analyst from the merely mathematically literate student. Only a small percentage of the students who begin to undertake study in research methods ever aspire to actually “do statistics”; rather, their aim is to become competent data analysts, the overwhelming bulk of which involves effective application of relatively simple procedures looking at central tendency and variance, particularly the latter. It would be far better to give students a solid grounding in relatively basic techniques than to push them toward more complex analytics which are, frankly, very often misapplied. A little knowledge can indeed be a dangerous thing in quantitative analysis.

The most substantial part of my own research education came through essentially apprenticing myself to a group of highly trained and well experienced researchers at the University of Michigan. Courses and workshops focused on specific kinds of techniques were helpful, but mainly in terms of providing working vocabulary and basic understanding of underlying principles. Learning what to do, when, and what to do when you run into dead ends in the context of real meaningful data analysis, not simply examples ginned up for pedagogical value, is essential to effective research practice down the line.We should be expanding opportunities for research internships and practical learning wherever possible.

Then, too, we need to recognize that a substantial portion of students in such classes are there purely for the instrumental value of certain techniques in the completion of a dissertation undertaken primarily for purposes of professional advancement. For a substantial portion of the students currently engaged in doctoral study across the board, the last statistic in their dissertation is the last statistic they will ever meaningfully employ. It really does no one any favors to push people like this through multidimensional scaling and structural equation modeling. One can obviously debate the relative wisdom of extending the PhD degree to those without serious research aspirations, but that battle was lost a long time ago.

All this suggests that it might be better to think in terms of training potential researchers in a set of things called “applied data analysis” rather than a set of things called “statistics”. My own most effective teaching has been where I could combine statistical analysis with research design and methods in an integrated fashion, so that techniques were learned in the context of data and data management. It’s really much more important to teach most students when they should not be using certain kinds of techniques than the minutiae of those techniques themselves. Certain students will always find themselves more drawn to the statistical end of things, and should be given full opportunity to pursue such interests. The vast bulk of students would be better served with a focus on good data analysis, grounded in just enough mathematics, than by heavy concentration on advanced techniques which they probably shouldn’t be using anyway.”

I’ll have more to say about this soon.