Recently, there has been extensive renewed interest in the question of causality in social systems. The question has been particularly anguished in the analysis of randomized clinical trials – those very expensive and presumably scientifically sound procedures that presumably tell us what drugs work and what drugs don’t. Coming on the heels of a seemingly endless series of studies each of which seems to contradict the last one – coffee is a killer; coffee will save your life; etc. – we now have a painful examination of the whole idea of a clinical trial, and indeed of the overarching idea that it is possible in the behavior of humans and their societies to establish causality at all, at least in the sense that physical scientists would use the term: “If A, then B”. This is a really interesting and complicated question, and it’s going to take several posts at least to explore various parts of it.
I don’t propose to go into extensive detail on the entire philosophy of causality; there are plenty of sources to provide that. Let me summarize what I believe are the major conditions for establishing a relationship between two things that most people would be prepared to accept as reasonable evidence of a degree of causality:
- A precedes B in time
- There’s some logical and understandable mechanism by which A might be presumed to cause B
- Most of the time A happens, B follows
- There’s nothing apparent that might be expected to cause both A and B
A and B here are representations of theoretical constructs or ideas – things we can’t study directly, because they don’t really exist. That doesn’t mean we don’t try. Aside from exploratory qualitative research, most research comes down to examining relationships among such theoretical constructs by measuring variables that correspond to those constructs and looking at statistical correspondences among the variables. “Constructs” are theoretically interesting ideas about how behavioral phenomena are lumped together. They can’t be measured directly, but they are assumed to have a variety of consequences that will show up in attitudes and/or behaviors that can be measured. Theories are often made up suggesting relationships among constructs rather than of relationships among concrete and measurable phenomena. But we don’t have tools for concretely measure relationships among constructs — all we have is statistics, and it has to work with variables, or mathematically represented images of those constructs.
Statistics is, as we have said, a great tool for untangling relationships among relatively well-defined phenomena. This is because almost all statistical procedures come with fine print attached in the form of assumptions, most specifically about the nature of the data to which they are being applied. These involve the distribution of the variables, the degree of relationships among them, the nature of the sampling procedures, and a variety of other specific aspects, most of which would be impossible to comply with under the very best of circumstances, and real-world research never represents anything close to those. Thus, in varying degrees, we are always a little shy of full compliance with the rules of statistics. Since the conclusions we tend to draw are usually fairly general, this doesn’t really matter in most cases.
However, whenever these kinds of situations operate in life, we tend to find activities focusing on the border. The dialogue becomes, “Well, we got away with that one; now can we push it just a little further?” Thus, we are faced with increasingly touchy problems as we deal with more and more complicated data, less well-defined questions, and more complex measurements and statistics.
We have consistently been taught that it is only through true experimental designs that we are able to make reasonably certain statements about cause and effect relationships, and even then it’s dicey. But the fact is, the overwhelming bulk of both evaluation and market research conducted in this country takes the form of cross-sectional surveys and observational studies, structured as “quasi-experiments” at best, rather than the so-called true experiment. Given the widely documented weaknesses of behavioral science experimentation, quasi-experimentation is probably not a lot less systematic than what gets billed as true experimentation — but it does put considerable pressure on researchers to extend their degree of confident inference about causal relationships at least up to, if not necessarily over, the line permitted by the scientific model.
Why? Because we are desperately seeking for information about causality. Although we ceremonially chant the mantra of “correlation does not equal causation”, we also use terminology such as “predictor” and “criterion”, interpret certain coefficients as “effect sizes”, and use the technique as evidence to support what are in effect causal models. The language is all about causation, because that’s where the money is. Co-variation of two phenomena is a pretty weak property; all it means is that two things change at the same time, which is not real hard to bring about. The really interesting thing for us humans is to have things change because we want them to. Implicitly or explicitly, the purpose of most of the research we do is to figure out ways to make that change process work better or more efficiently — to find better and faster levers to pull to make things happen. There’s nothing wrong with this, either in terms of motives or results. Just knowing that things hang around together on street corners may be interesting, but it’s not why we got into the business. Despite our chanting “correlation is not causation”, in practice we don’t really believe it. The real money is always going to be in causation, not simply association. We’re all at bottom control freaks. Once upon a time, having control saved us out there on the savanna from being eaten by lions, and we’ve never forgotten that.