It does a grumpy teacher’s heart good to see the recent New York Times Magazine piece on student evaluations. It turns out that the skeptics are right: evaluations are popularity contests that say little about whether students have progressed.

For fifteen years, I taught the basic psychotherapy course in a psychiatric residency program. I got some good evaluations. Often, I got bad ones. I expected as much. I ran a graduate school seminar, but for doctors-in-training for whom time was in short supply. Worse, I was teaching a subject that elicited strong responses. Half the trainees thought therapy was nonsense; they believed in monkey spinal fluid. The other half had gone into psychiatry to do psychotherapy and were discovering that the field had drifted elsewhere. And here I was, with the whole group, demanding attention to complexity.

I had devised a course that exposed residents to a range of therapies. My approach involved assigning a reading about how to conduct a given treatment; in the class hour, we would then infer the psychological principles and assumptions that must underlie the recommended procedures. We moved backward, from practice to theory. My idea was that therapy is always changing. I wanted my students to develop analytic tools, so that in the face of a new intervention, they could say — oh, that’s just a cognitive (or strategic, or self-psychological) treatment in a different guise.

I hope that I was enthusiastic about the material and encouraging to the participants; in return, I expected effort. To buffer my toughness, I always co-taught the seminar with someone kinder and less exigent, a colleague with expertise complementary to mine, in anthropology, history, or psychology. Generally, the co-teacher would get high marks. My idea was that this nurturance freed me to set rigorous standards. My goal was to have students look back ten years out and say, “That little course really helped.”

As I say, I the evaluations were all over the place, but with plenty of negatives. To its credit, the residency program simply absorbed the criticism, perhaps because subsequent teachers found the residents well prepared. Parenthetically: as other demands on residents increased, and as entering students began to arrive with less background in psychology (though more in biochemistry), I made the curriculum easier and the reading list shorter — steadily, every year, for fifteen years. Even so, at the end, residents found the material difficult. It was. I wanted trainees to see that bright people had thought creatively and written passionately about how one mind influences another. I left room for agnosticism, but at the same time, I tried to demonstrate that, like most of what the students would encounter in medicine, psychotherapy was a technology.

As I say, I like the Times piece. Midway through, it features a curmudgeon like me. He values his discipline, philosophy, and he is loath to compromise on his didactic methods. Pressured to improve his score on student evaluations, this teacher lets his students know that they will all get A’s. Presently, he has the highest evaluations in the department. He explains to the Chair (and here I take some liberty with the anecdote): Better to pander to students through the grading system than to pander to them in the instruction.

Reading the Times summary, you lose all respect for student evaluation as a measure of teaching quality. The assessment tool has no power to predict how students will do in subsequent classes. Students who say they have not learned soon prove they have — and at the same level that they learned in classes where they praised the instruction. This result is astonishing. You would imagine that assessments would pick out the egregiously unqualified teachers and that the identification of those bad apples would color the overall results. It doesn’t. Teachers with low marks perform as well as teachers who earn praise.

Evaluations seem to measure charm. When a psychiatrist-researcher had an affable actor spout nonsense in the guise of a scientific lecture, both professionals and students gave the talk high marks. When students were offered chocolates before filling out forms, they gave favorable ratings, when compared to students in the same class who got no candy.

The lesson from education — preferences say little about benefits — probably carries over to psychotherapy. The great Hungarian-born psychoanalyst Franz Alexander said that if the patient did not end by hating the analyst, very likely the treatment was a failure. This quip may contain only a partial truth. Studies show that, in simple and brief treatments at least, the therapist-patient match is a good predictor of outcome. Still, I can think of patients who complained mightily and fared well.

In education, hiring or promotion often turns on student assessments, the ones that don’t correlate with learning. Increasingly, third parties use similar satisfaction measures to judge psychotherapists. Consumerism has its benefits. Both teachers and doctors were once free to humiliate their charges; thankfully, that standard has changed. But the teaching evaluation literature suggests that the consumer model has its shortcomings, as when it emphasizes niceness at the cost of results.