I’ve been engaging in a bit of Facebook debate with an old college friend about statistical prediction and generalization. Here’s a summary of my side of the discussion. In general, I’m arguing that while prediction on the basis of categories is useful and valid at some level, it also breaks down critically when decisions have to be made about individual cases. Here’s why, and it all starts with the origin and purpose of categorization in general. Sorry for the length here, but I am going somewhere with this. Bear with me.
Animals in general developed the ability to categorize things – to lump things together as though they were the same thing – in order to improve their chances of avoiding danger and accessing resources most efficiently. With categorization comes learning and generalization, which is much more effective cognitive processing than having to approach every event as sui generis. If you see a small dog, your perception of danger is likely to be a lot less than if you see a large dog. (In my case, since as a small child I was mauled by both a large dog and a small dog, I have a residual aversion to dogs of any size, but that’s just me.)
Note that size is NOT a good predictor of intelligence, or likability, or color, or any of a thousand characteristics that might interest you about a dog. Of course, people form all sorts of associations based on their experiences and on what they have been taught by others. Some people only like large dogs; others only small dogs. Some people believe that some breeds are inherently smarter than others, or that some are better with small children, or any of a thousand other propositions. Adherence to these propositions about associations among dog characteristics is occasionally supported by personal experience of varying sorts, but the vast bulk of it is learned from others. But it is none the less firmly believed although unsupported by direct evidence.
This kind of category processing works pretty well in the case of dogs, or zucchini, or rocks. But the more complex a thing is, the more relevant characteristics it has, the more they interact, and the more difficult prediction becomes based on limited information. Once we start dealing with human beings and even more so, with human systems, category processing quickly becomes technically unmanageable. Without fairly deep knowledge about a person or organization, we aren’t able to make very good predictions about anything. First-order approximations, yes; and for many purposes they work. But their only justification is that they often work; we can’t say why they work. And we know deep down that they aren’t probably correct. But category processing is so hard-wired into human consciousness that it’s where we intuitively and inherently go when faced with the unfamiliar.
Some characteristics are much more easy to determine than others. Things like presented gender, size, color, activity level, and the like are relatively easy to perceive at a distance, and thus form the basis for most initial categorizations of other people. We all have our own set of associations of these characteristics with a whole lot of other characteristics, things that we’re probably much more interested in such as likability, personality, intelligence, and compatibility. At that level, associations are highly personal. Some people have an immediate positive association of height with other good things; some people immediately associate height with negative things. As in the case of dogs, some of these associations are based on experience; a lot of them are based on things we’ve been taught or just picked up. A single negative experience can cause you to form a whole body of negative associations which are the most part untrue. My own association with dogs is a good illustration. I haven’t been mauled by a dog since 1945, but I still carry that association with me.
Where we often get in trouble is when we have to make decisions about a particular individual and about dimensions like personal interaction, as in job hiring, being a babysitter, getting laid, or a whole variety of other things. This is where categorizations can get in the way. As you say, making predictions based on experience with categories is statistically a good practice, but it’s often misleading in the individual case. While it may be statistically true that men overall are better at coding than women overall, it doesn’t make much sense not to interview women for coding jobs, because the next woman to walk through the door might actually be better than the guy she was sitting next to in the waiting room. Only by examining her own credentials – that is, giving her a chance through the door – will you ever find out what’s true. And if you decide to simply exclude women because of their statistics, you are likely to miss out on a great many excellent coders and employees.
So I’m not arguing against statistical prediction as such, simply cautioning about its inherent limitations and in particular its own unhelpfulness in assessing individual situations. Since so much of national policy these days seems to be based on generalizations about groups and identities, I think it’s important to remember these essential limitations on the use of categories and identities.