Verified by 
This time of year, faculty are often busy writing letters of recommendation for students applying to graduate school and responding to reference requests from potential employers. Considering the amount of time and effort spent writing letters, completing reference forms, and providing phone references, it makes sense to ask whether letters of recommendation actually predict student or employee performance. Given that one of the basic beliefs in psychology is that the best predictor of future performance is past performance, one would think that references would be valid predictors.
Unfortunately, a meta-analysis that Felice Williams and I conducted suggests that although the relationship between references and academic and work performance is statistically significant, it is of a relatively small magnitude. Based on 51 research samples, covering almost 11,000 students and employees, we found that the average uncorrected correlations between letters of recommendation/reference ratings and work and training performance are .17 and .25, respectively.
Although these correlations are similar to those found with many selection techniques, our data at Radford University indicated that reference ratings and letters of recommendation did not add incremental validity to the combination of GRE scores and undergraduate GPA.
Why isn't the relationship higher? One reason is that few references are negative. In fact, two studies indicate fewer than 7% of students or job applicants receive average or below average reference ratings! This is especially true when students don't waive their right to see their references. Thus, letters of recommendation might differ in the traits or characteristics used to describe an applicant, but will seldom differ in how the referee evaluates the quality of the applicant. Given that most applicants select the people who will be their reference providers, we probably shouldn't be surprised by the lenient ratings.
Reference providers may also avoid giving low ratings or providing negative comments out of a fear that the applicant will be angry at them or the reference provider will be the target of a defamation suit. A person providing references can be charged with defamation of character (slander if the reference was oral, libel if it was written) if the content of the reference is both untrue and made with malcious intent. Although successful defamation lawsuits involving references are rare, the potential for a lawsuit often results in organizations limiting the scope of their references to confirming dates of employment and salary levels.
A second reason that letters and references are not good predictors of future performance is that two people providing references or writing letters for the same person seldom agree with one another. On the basis of only eight samples across six studies, we found that the average reliability for references is only .22. In fact, our research suggests that there might be more agreement between recommendations written by the same person for two different applicants than between two people writing recommendations for the same person. Thus, letters of recomÂmendation may say more about the person writing the letter than about the person for whom it is being written.
One reason for this lack of agreement is that two reference providers might actually see an applicant in very different situations. For example, a student's academic advisor might see different behaviors than the student's classroom professor or research advisor. The academic advisor might comment on the student's reliability and career plans whereas the professor seeing the student in the classroom might concentrate more on the student's grades and writing ability.
Given the problems associated with references and letters of recommendation, what are potential best practices? Based on limited research, it appears that content-valid, structured approaches work better than generic reference checklists. That is, rather than asking about broad traits, ask specific questions related to the desired behaviors. For example, rather than asking if the student is reliable, ask about specific behaviors such as keeping advising appointments or attending class.
When reading letters of recommendation, keep in mind that much of the content is more of a reflection of the writer's style rather than it is an indication of the applicant's ability. That is, a short letter of recommendation might reflect the terse writing style of the professor more than it does a "hidden message" that the professor does not like the student. Likewise, a letter of recommendation that does not mention the student's personality probably indicates that personality is not important to the writer rather than the "hidden message" that the student has a terrible personality.
When writing letters of recommendation, research indicates that readers prefer to see concrete examples rather than general statements. Although it should go without saying, avoid statements that might draw unusual attention. The strangest statements I have seen in letters of recommendation are:
- He is cuter than a baby's butt
- She has no sexual oddities that I am aware of
- I/O psychology has long been dominated by white males and accepting Karen will give you an opportunity to rectify that
- You previously accepted two of our mediocre students so I can see no reason why you wouldn't accept Bob.
