Elysian Excellence

Would you set your own cut-off for success?

Posted Aug 28, 2019

     I wrote down the grades I wanted in every class. —C. Booker

     The worst thing I can be is the same as everybody else. I hate that. —A. Schwarzenegger

A few years ago, I took a look at the phenomenon of grade inflation, but left open the question of whether students want it (Krueger, 2015). Perhaps they do. Surely, they wouldn’t mind getting better grades, but if they got what they wanted, grade inflation would be the unfortunate consequence. What students really want, it seems, is to distinguish themselves from their peers. What matters is to have a better grade than their classmates. If most feel this way, we have a collective action problem of the type of the prisoner’s dilemma. Pushing for a higher grade for oneself, e.g., by petitioning the instructor, is an act of defection. The benefit to the pusher is less than the sum of the damage to everyone else.

Learning that the prospective percentile for the grade of A is, say, 20 percent, the student will be concerned about the high probability of receiving a B or worse, and thus worry about a declining GPA. This student will want to argue for a more relaxed cut-off, perhaps 40 percent or even 60 percent. Ultimately, however, most students will be unhappy when the instructor declares that everyone who shows up will get an A.

Students, and the bureaucrats who manage them, expect instructors to set fair grade cut-offs. How this may be accomplished receives less thought. Here, I explore the possibility of relieving the instructor from this onerous task and putting those who seek fairness in charge—the students, that is; the bureaucrats can return to weightier matters.

The idea was inspired by a story Dixit & Nalebuff (2008, p. 290) tell in their chapter on coordination problems. At the mythical law firm Justin-Case, no junior partner has been promoted in years. The juniors are unhappy. The senior partners respond by ranking the 10 juniors and conveying each rank privately to its holder. Then they ask the group to decide by majority vote on the cut-off rank for promotion. Crucially, they allow repeated voting. After beginning with a consensus vote that all juniors should be promoted (cut-off = 1), the better-than-average motive asserts itself and dissent emerges. The more highly-ranked juniors notice that their excellence is no longer marked. They propose a slightly higher cut-off and receive majority approval. With repeated voting, the majority-supported cut-off rank creeps up. With each increment, only the person who fails to be promoted under the revised rule dissents. When the bar passes the rank of 5, it continues to get majority support because now the low-ranking juniors realize that they are better served by an even higher cut-off so that their failure is no longer unique. In the end, the group votes to retain the old system of no promotion.

This dismal outcome stems from the use of the majority rule and its sequential application. If each junior privately proposed a cut-off point, it would presumably be their own rank, in which case averaging would amount to range-splitting.

The story generalizes to the classroom. Each student will want to be included in the A range, with the B range starting just below. This arrangement satisfies the self-enhancement motive (Alicke et al., 1997). However, with the repeated use of the majority rule, no student will be left with an A. This scheme reveals the unintended and negative consequences of exercising students’ preferences; it undermines their interests and the educational mission. 

It is, however, possible to plumb students’ desires and intuitions without allowing them to set policy. Specifically, imagine students are asked at the beginning of the semester to predict their own end-of-semester percentile standing in course performance, and to suggest a cut-off percentile for A grades. The two estimates will probably be positively correlated, and predicted own performance will likely be higher than the proposed cut-offs – if only by a bit.

Adding the final course grade and the students’ GPA to the data allows for tests of some interesting hypotheses. Here is a sample:

  1. Good students (high GPA) predict and receive higher grades. This should be so in a partially coherent world.
  2. Predicted grades are more strongly correlated with performance than are GPAs. This should be so if students have valid knowledge about their effort and ability in the subject matter. 
  3. Proposed cut-offs are correlated with the other three variables.
  4. The correlation between proposed cut-off and GPA remains positive when predicted and actual performance are controlled. If so, general ability affects conservatism in grading. By reverse inference (Krueger, 2017), we may assume that those students who propose stricter grading are the more able ones.

The last prediction has an interesting implication. Instructors seeking to maximize the number of talented students in the course, could – in theory – admit students on the basis of the grading scheme they suggest.

There is no consensus on what makes a grading scheme fair. Perceptions of fairness are notoriously tainted by self-interest (Thompson & Loewenstein, 1992). What is an instructor to do? A useful heuristic is to avoid the extremes. If most students in Dr. Markman’s course receive an A, they will dismiss the course as a gut. The most talented students will stay away because they want their grades to signal positive distinction. If Markman announces that few students will receive an A, some of the most talented students will also stay away because of the lower probability of distinguishing themselves.

There is, then, no easy way to attract a group of above-average students through the manipulation of the grade scheme alone. Perhaps a renewed focus on the course material might help. Yet, even the composition of the course material will raise some of the same issues. If the material is easy, many talented students will stay away. If it is hard, these students will worry about not getting an A. Some relief may be found by cultivating a positive correlation, over courses, between the toughness of the material and the leniency of grading. The best students will seek courses challenging enough to discourage their weaker peers, but offering a likely A if a reasonable performance threshold is passed. I suspect, however – with only anecdotal evidence – that this correlation tends to be negative on many campuses. That is, the easy courses offer more As. This correlation, if it exists and if it is correctly perceived, allows students to identify courses that are easy and yield a good grade, which is a state of affairs that debases the currency. The talented are punished if they work harder at greater challenges, while risking lower grades.

We must distinguish between the case in which the instructor sets the grade cut-off and the case in which the students do. Talented students privately propose stricter cut-offs, but reject these same cut-offs when proposed publicly by the instructor because this drives away too many of those students from whom they wish to distinguish themselves.

The lesson Dixit & Nalebuff sought to convey was that leaving the setting of promotion or grading standards to those who are affected by them creates a dilemma that these same individuals cannot want. The senior partners' or the instructors’ dilemma is that their strictness and consistency (no special deals) might be misunderstood as a sign of inflexibility or lust for power.

Reed only

Reed College, in Portland, Oregon, the “Brown University of the West Coast,” does not assign letter grades, at least according to popular belief. Read Reed’s own statement regarding Evaluation of students. It tells a more complex story: Letter grades are recorded and archived, and they can be accessed by interested students. Educational emphasis, however, lies on students’ engagement with and mastery of the material, their self-assessment, and detailed feedback from instructors. Reed tries hard to uphold an educational ideal under siege in the age of bureaucratic accountability and "data." Reed is highly respected and it attracts excellent students. Two generations ago, Brown students defenestrated general curricular requirements, but left the grading system untouched. Reed went the opposite way. Its freshmen must read Homer, and self-assess how well they did. Students at which university are more "intrinsically motivated," if such a thing exists (Krueger, 2019)?

Homer bound

Τῶν δ' ἄλλων ἐμέ φημι πολὺ προφερέστερον εἶναι,
ὅσσοι νῦν βροτοί εἰσιν ἐπὶ χθονὶ σῖτον ἔδοντες.

I far excel every one else in the whole world,
of those who still eat bread upon the face of the earth.

     —The Odyssey, book VIII, 221–222, translated by Samuel Butler

Pass/Fail

A reader offers the following solution: I always felt that grades should be objectively based on mastery of the material. Criteria are set at the beginning of the course and grades awarded based on reaching set benchmarks. Everyone in the class could receive an A or no one - completely dependent on students' achievements. The benefits [sic] of such a system is that reduced competition between students may lead to enhanced cooperation and enhanced learning.

Offering an A or nothing amounts to a Pass/Fail scheme. Brown University offers this option, but few students use it. Dixit & Nalebuff note that the choice of the pass/fail option signals weakness. Students realize that if they pass a course, readers of their transcript will infer a lack of confidence and a likely grade of B or C if it were given. The pass/fail design leaves the material’s difficulty as a wild parameter. A course can be designed so that only the top 10 percent of students will pass it, or so that the top 90 percent will pass it. We can ask students to vote on the material’s difficulty, and we are back to Justin-Case.

The types of pass/fail tests the reader describes do exist. Take, for example, the test for the driver's license or the test for citizenship. Here, the bar is low. The successful student demonstrates adequate, if minimal, competence. These tests can be made harder, but it remains to be seen if students will take this as invitation to cooperate with one another. 

References

Alicke, M. D., LoSchiavo, F. M., Zerbst, J., & Zhang, S. (1997). The person who out-performs me is a genius: Maintaining perceived competence in upward social comparison. Journal of Personality and Social Psychology, 73, 781–789.

Dixit, A. K., & Nalebuff, B. J. (2008). The art of strategy. New York: Norton.

Krueger, J. I. (2015). Grade flation. Psychology Today Online. https://www.psychologytoday.com/us/blog/one-among-many/201508/grade-flation

Krueger, J. I. (2017). Reverse inference. In S. O. Lilienfeld & I. D. Waldman (Eds.), Psychological science under scrutiny: Recent challenges and proposed solutions (pp. 110-124). New York, NY: Wiley.

Krueger, J. I. (2019). Trinsic motivation. Psychology Today Online. https://www.psychologytoday.com/us/blog/one-among-many/201905/trinsic-motivation

Thompson, L., & Loewenstein, G. (1992). Egocentric interpretations of fairness and interpersonal conflict. Organizational Behavior and Human Decision Processes, 51, 176– 197.