Verified by 
The backbone of corporate recruiting consists of resume reviews, job interviews, and reference checks. If this backbone was part of the human skeleton system, we would say that this backbone suffers from calcium deficiency. In this article we focus on reference checks: why the system is flawed and how you can improve it.
What’s Wrong?
A 1984 meta-analysis about the predictive validity of reference checks suggested that reference checks have somewhat higher value than job candidates’ self reports. But the difference is not significant. Predictive validity was operationally defined as agreement between reference data prior to hiring and supervisor performance evaluations a year later. (Schmidt & Hunter,1998).
Given the time and cost involved for reference checks, a “somewhat” higher validity than self-reports is not exactly a vote of confidence about reference checks.
In this sense, we see a problem with the entire enterprise of reference checks.
We see four issues that create problems for reference checks: risk management, peer-to-peer conversations, competence, and community. Following a discussion about these problems we will provide recommendations you might consider implementing in conducting reference checks.
Risk Management
When someone calls an official at a company to ask questions about a former employer, that official may or may not be put into a risky situation. The United States is one country where risk tends to be on the high side. There could be circumstances where candidates will see references provided and can sue the former company as well as the official who provided the reference.
To manage these real risks, some of our client companies require officials to only provide reference checkers with dates of employment and whether the departure was a termination or resignation. We all know how many official “resignations” are really terminations.
In other words, if you go through the official channels you may get nothing but bland information.
And if you call people who no longer work at a company, they may also be concerned about personal liability. They may refuse to answer or provide bland information. We have seen several cases where people provided us with deliberate misleading information.
Wrong Person Asking the Right Questions
What is the status of the person asking the reference questions? We find that law firm managing partners will be more honest with other law firm managing partners than they will be with professional reference checkers. CEOs will be more honest with other CEOs.
Many recruiting firms outsource the reference check to a junior person. This can be a mistake. We have firm partners do the calling but not the partner who did the search. We want a fresh perspective. We want the person getting the call to know that they are speaking with a partner.
Sometimes we recommend that the hiring authority be the one to make the call as she is more likely to get useful information. Peers will be more open with peers.
Competence
There is variability in the competence of those tasked with obtaining references. Structured reference checking questions and scoring procedures have shown predictive validity when used with entry-level candidates. (Taylor et al, 2004).
In general, if you ask a general question, expect a general answer. For example if we ask, “What is this person like to work with” we might get a vague answer like, “A Real Professional.”
We recommend that specific questions be developed and written down in advance. All people interviewed should get the same questions in the same order.
Community
We have seen situations where reference providers deliberately provided false information to us about candidates. For example, one reference said of the candidate that his greatest weaknesses was that he needed to be “coaxed into honestly speaking his mind.” Once he was hired, however, the problem was to get the VP to stop talking.
The person providing the reference may no longer have a reporting relationship to the job candidate but they still are part of the same community. Why burn bridges?
The implication is that you want to talk with several people beyond the names provided by the job candidate. You want to hear the same themes over and over again. It is not important what one person says in a reference check. What are important are the themes that are uncovered from multiple people interviewed.
What References Cannot Do
Be appropriately humble about what reference checks can and cannot accomplish. It probably is not a useful predictor of technical competence. Better measures of competence would be work samples brought in or giving cases/vignettes to solve.
For example, the U.S. Foreign Service wants its officers to be able to appropriately organize information and to coherently write in English about what they observe. Reference checking is a poor way to get that information. A more effective approach would be to provide candidates a short case full of relevant and irrelevant information. Candidates must next write a memo describing what happened. The written products are graded on the quality of writing and the ability of candidates to distinguish facts from suppositions. In other words, if good writing ability is important, then provide candidates an opportunity to write well.
If understanding a balance sheet is important, create a short case with financial information and ask the candidate to provide an analysis.
References do provide useful information about communication style important for job success. Three of these behaviors are Conscientiousness, Openness to New Experiences, and Corporate Citizenship.
Conscientiousness. This is the most desired employee characteristic world-wide. Reference checks have proven validity in capturing this important attribute (Hunter & Hunter, 1984). Consider asking the following question: “This job requires a high level of conscientiousness. And that means a “you can count on me to get the job done” attitude. Can you tell me a story in your direct experience of this candidate demonstrating conscientiousness?
Openness to New Experiences. Being innovative is an increasingly important value in many industries but not necessarily in all jobs. Certain jobs are best done with people who enjoy repetitive and predictable work. There is an assumption that as people age, they are less open to new experiences. But that may not be valid. Young people can be closed to new experiences that older workers might embrace. Consider asking the following question: “This job requires a being open to new experiences and discarding well learned habits of past success. Can you tell me a story in your direct experience of this candidate about how the candidate was open to new experiences?
Corporate Citizenship. If work is accomplished alone, then corporate citizenship is irrelevant. But if work is performed in teams, then citizenship becomes a critical ability. We have yet to meet a job candidate who does not boast about “collaborative skills.” Of course, that is a fuzzy concept. It could mean: “what’s good for me is good for the company. Therefore I am collaborative.”
Corporate Citizenship is a more precise concept. Consider asking, “Corporate means doing something for the good team or the company that might not benefit the individual or her functional area. Corporate citizenship can mean an airline employee helping a passenger change a flat tire in the airport parking lot. Can you tell me a story in your direct experience about how the candidate has demonstrated citizenship behavior?
Tell me a Story
You will notice that in the three examples we have provided, we have asked references to tell stories
The stories need not be about workplace situations. Thus a mother who has not had a paid job for seven years might still have wonderful stories about Conscientiousness when she was Den Mother of a Cub Scout unit.
References are one of the backbones of the corporate recruitment system. But that backbone has severe calcium deficiency for reasons outlined in this article. We have suggested some ways of improving reference checks.
Our theme is to ask specific questions for specific answers. And expect general responses if you ask general questions. Indeed, if you ask did a reference check about us using general questions you will probably find that we are trustworthy, loyal, helpful, friendly, courteous, cheerful, thrifty, brave, clean, and reverent!
References:
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological bulletin, 124(2), 262.
Taylor, P. J., Pajo, K., Cheung, G. W., & Stringfield, P. (2004). Dimensionality and validity of a structured telephone reference check procedure. Personnel Psychology, 57(3), 745-772.
###
