Another debate about the validity of social research

By | June 21, 2013
thinktank4

I must confess that this is another column based on some exchanges with other colleagues on a Chronicle of Higher Education discussion site – this one specifically discussing issues relating to intelligence and its relationship to wealth. The original article suggested that rich kids were increasingly in a position to take advantage of their privileged background to dominate the educational testing sweepstakes. The discussion fairly quickly devolved into a debate about the degree to which intelligence might be inherited and the likelihood that the rich, being more intelligent, might thus be dominating educational testing by virtue of having smart kids rather than considerable money.

Being the contrarian that I am, I responded as follows:

There is extensive debate on the heritability of intelligence (however that is defined), the conclusion generally being that so many factors are involved it’s virtually impossible to find a sample large enough to partial out all the variables that affect such intergenerational transfers. Your inference appears to be that if parents make a lot of money, they must be intelligent, Since the parents are intelligent, their kids will be intelligent too, and will succeed in school; then, by virtue of your first assumption, they too will go on to make a lot of money, thereby proving their intelligence in the next generation. Your inference is supported only by systematically suppressing all the other variables that might affect such an equation, including among other things the systematic rigging of the educational system in favor of rich kids.

There have been many studies of twins raised either together or separately. It’s clear that having identical DNA at conception induces a number of similarities to a degree that would not be assumed by chance alone (among other things, if one twin is gay, then the odds of the other twin being gay are significantly enhanced, even if they were raised completely separately.) But it’s also true that almost nothing is purely inherited or genetically determined throughout; the interplays with environment are complex, and only get harder to trace the older people get. Even the DNA of identical twins is subject to changes throughout life; even if they begin identically, they may be somewhat varied unpredictably as early as age 5.

The progress of good science always depends more on the generation of good questions than on the assembly of “answers”, which are always tentative and subject to reinterpretation. The surer the answers are proclaimed to be, the greater the likelihood that they result from junk science rather than the real thing.

After being chided by a reader observing that the connection between wealth and intelligence seemed fairly self-evident, I again took exception here:

As a researcher, I am obliged to be a permanent skeptic, reasonable or not – and as such, I am never allowed to believe that I have found “truth”, particularly a “truth” that allows us to forego further inquiry. I certainly agree with you on observer bias and on the “theory/data” problem; far too much research is ideologically driven, consciously or unconsciously, and pointing this out can sometimes entail professional suicide. Part of learning to be a good researcher is learning to detect such biases in the studies of others; unfortunately, this doesn’t always translate into an ability to detect one’s own.

I would agree that better family circumstances can lead to better test scores (for whatever test scores are worth), and that more income makes it easier to take care of kids. Between me and my ex-wife, we made enough money to give our daughter an excellent start, including some first-rate private schools (of course, she was brilliant to begin with, but it helped.)

But – and this is perhaps my main point – possession of money in large quantities exerts power not necessarily justified by either the genes, intelligence, or natural or exercised talents of those possessing it. In short, being rich doesn’t necessarily mean that you’re smart, but it does allow you to attain the privileges normally reserved for those who are, and not infrequently to use those privileges to the disadvantage of those from less wealthy backgrounds. I believe that this assessment is supported by good research.

The problem is that …rich kids tend to behave like other rich kids, regardless of which of the three categories (smart, lucky, or inherited) their parents might have come from. For the rich kids, test-taking intelligence is about the only kind they’ve had the opportunity to develop. And the rich have had the power to jigger the system so that the one thing that their kids are good at becomes the most determining factor for educational and thus early career success. It’s a neat trick, no doubt, although Further Research Is Needed, probably starting at the Beverly Wilshire Hotel bar.

I doubt that many minds were changed in the course of this dialogue, but I have taken it upon myself to continually try to defend quality behavioral science research from both those who would deny its fundamental validity and those who would over eagerly embrace whatever the latest flavor of the day might be. I’ve tried to stress that no proposition in the natural or behavioral sciences is ever “proved”; proof belongs exclusively to mathematics and formal logic. Causal propositions are suggested; there are formal rules dating from JS Mill about how causality can be suggested and tested for. A proposition suggests hypotheses, or statements about relationships among data elements, that can then be tested for consistency, assuming that you have relevant data of good quality. In this process a hypothesis may be either supported or falsified. Support for a hypothesis is always tentative pending the completion of more research and the collection of more data that may eventually falsify the hypothesis. The status of the general proposition may be damaged by the failure to support hypotheses derived from it (if the hypothesis relates to a key element) or merely embarrassed (if the hypothesis is minor and not related to key elements.) However, there is a long way to go in establishing the acceptability of this behavioral science research model, even among Chronicle of Higher Education readers.

  • http://rrp.stanford.edu Rega Wood

    Hi JD: Thanks for sending a note to Reed; I once spent a hour or two looking for an email or snail mail address for you — without success. Allen and I are now retired from Stanford and teaching at Bloomington, IN, for Indiana University. Every summer we return to California from Indiana, and it would be lovely to meet some time. I’ve enjoying poking around your impressive website. Good luck with construction! Rega

    • DrEvel1

      Great to hear from you, Rega. I was just thinking about Allen this afternoon. So what we have here is a major Jungian synchronicity! Actually, I am generally pretty googlable (if that isn’t a word yet, it soon will be.) Trading Palo Alto for Bloomington must have been a bit of a shock, particularly come January. Of course, you spent all those years in upstate New York and New Haven, so perhaps not. I couldn’t return to that sort of climate again even if I were to be offered all the tenured chairs in the Big 10! I hope that we can arrange an opportunity to reconnect!