Judging Perfection (Part 1)

By | September 25, 2016

Women’s gymnastics is one of the most popular Olympic events, and with good reason. It’s truly amazing the kinds of things that people can do with their bodies, twisting, rolling, hanging in the air. But the essence of the Olympics these days isn’t really in watching the sport; it’s watching the scores. Although you’re watching the top fifty or so athletes in the world, which is an amazingly small number, it’s really only the top ten or so who get any attention, and after the competition, it’s really only the medalists who are remembered outside the coterie of the sport. This is both sad and frustrating, but it’s the nature of today’s sports/media. Be the #1 person in the world and get noticed; be only the tenth best, and no one cares.

Slate recently (August 8) published an interesting video entitled “Everything You Need to Know About Gymnastics Scoring; Or, why Simone Biles won’t get a perfect score in Rio.” Now Ms. Biles is undoubtedly either the best in her field in the world or within two or three of it on any given day. What’s particularly interesting in this video is the way in which judging is carried out at this level of perfect performance. As it turns out, Ms. Biles has a consistent habit of slightly crossing her feet during her amazing stunts in the air – something that only a slow-motion camera or an enormously experienced observer of performances that take perhaps three seconds to complete would ever be able to detect. But once that is detected, it guarantees that at least a tenth of a point will be taken off her score. At this level, scoring is not a matter of adding up the points for the things you do right; it’s a matter of starting with a perfect score and then deducting fractions of points for anything that can be noticed by a judge. The judges begin with the assumption of a perfect performance, and then take it down from there. It’s probably pretty much the same in any other sport where individual performance is rated by a panel of experts. Note that this only works for perhaps the best 20 or 30 athletes in the world any given sport; but it’s strikingly different as an assessment model from that generally applied to athletic performance.

So what’s the relationship here to education? Thinking back on all the thousands of papers and examinations and exercises that I’ve graded over the years for students in classes ranging from Whatever 101 to dissertation-level, I suspect that implicitly I’ve applied a version of this two-level standard – one standard for judging most students, and a separate standard based on perfection applied to those who have established themselves as excellent. In essence, two different kinds of rubrics applied to different kinds of students – undocumented, and until now, largely unnoticed.

Ever since I began teaching, I’ve been a strong opponent of grading on the curve. From the start, I thought it was ridiculous to assume that the distribution of performance in any given class would necessarily match the distribution of performance across all classes and situations. That is, one class might be full of really excellent performers and another full of those simply phoning it in. Trying to apply the same grade distribution to both of these classes would be grossly unfair. The excellent performers ought to be rewarded for their achievement in terms of the course goals, not penalized because they happen to enroll in the wrong section with a lot of other good students.

That said, there is obviously in any given class a range of performance. Some folks will get it quite quickly; others will struggle all term long just to understand the basic terminology and the key concepts. Early on it’s pretty easy to recognize which students fall in which categories. I did some analysis at one point on the grading habits of a number of adjunct faculty who worked under my supervision in sections of an MBA-level course. It turned out that the final grade in the course was almost perfectly predicted by the grade received on the very first assignment in the course. If you establish your reputation as an A student right from the start, you have to really try to screw up your performance in the rest of the course in order not to finish with that same A. if you start out as a B student, it’s almost impossible to finish the course with an A. it can be done, but only with enormous effort and a lot of personal attention.

The same applies to students more globally upon entering any program. I, along with most of my colleagues, was able to predict with a high degree of certainty whether or not a student was likely to complete our program successfully on the basis of the first one or two assignments that they completed in any one of our introductory courses. Completing any graduate program is much more a matter of personality and determination than it is of pure academic skill; it’s pretty easy to assess whether or not someone is bringing to the program the kind of attitudes and approaches that are necessary to success. This isn’t any kind of judgment on the moral worth of the people involved, or even whether or not they would benefit from the degree program itself. It simply reflects the fact that the attitudes people bring to any educational process are critical to successful learning, and they aren’t all that hard to notice.

(Stay tuned for Part 2)