The University of California and The SAT: Speaking the Truth?

In 2019, 3.7 million students graduated from high school in the United States. Approximately 60 percent of them, over 2.2 million high school students, took the SAT (National Center for Education Statistics). By comparison, in 1926, the first year the SAT was administered, only 8,000 students took the test.

Of course, a lot more than numbers has changed over the years. The current version of the SAT bears little resemblance to the original test. In fact, everything about it is different: the number of sections, the content of sections, the items, the scoring. The name itself has changed four times. Initially “The Scholastic Aptitude Test,” it is now simply “The SAT.” Every year, the test data are examined carefully and the test is revised accordingly. And over a century, the psychometrics developed and matured to formally study issues such as item selection, test construction, reliability, validity, and test bias. The current version of the SAT is informed by that work, nearly a century of scientific research.

In 1926, the SAT was developed by The College Board, and the project was spearheaded by Carl Bingham, a Professor of Psychology at Princeton University. Bingham was also on the advisory council of the American Eugenics Society, and his early work influenced the eugenics movement and anti-immigration legislation in the United States. Likewise, it is well known that the SAT, and the field of intelligence in general, has a legacy of racism. For decades, the SAT was biased against racial/ethnic minority groups, especially African-Americans.

Again, a lot has changed over the decades. Every year, SAT items and data are scrutinized for any forms of cultural bias, the test is revised on the basis of that work, and more research is conducted to improve the test further. Very few tests receive this type of close and long-term attention, and overall, this iteration process is how science works. It is cumulative. The SAT is now administered by The College Board and the Educational Testing Service (ETS), which is the largest private nonprofit educational testing and assessment organization in the world. ETS develops various standardized tests for both secondary education and higher education. Thanks to research conducted by ETS and academic researchers, the SAT has vastly improved over the years. Large amounts of data and many studies concerning the current version of the SAT suggest the test is not biased, the scores are statistically reliable, and they meaningfully predict college performance outcomes (Frey, 2019; Kuncel & Hezlett, 2010).

Despite this evidence, in 2018, a lawsuit was filed against the University of California, claiming that the SAT is biased against certain racial/ethnic minority groups and therefore the use of the SAT in college admissions is a form of discrimination. In response, the University of California assembled a special task force to investigate how the SAT is used to make admissions decisions at UC schools. The task force was asked to provide a recommendation as to whether the UC should continue to use the SAT or drop the SAT requirement for admission to the UC.

In February 2020, the task force submitted their report to the UC Administration and Board of Regents. It was also released to the public. I read the report in March and found it to be incredibly impressive. It is 225 pages and provides a comprehensive analysis of the SAT (18 years of data on thousands of UC students). The statistical analyses in the report show that the SAT is a statistically and practically significant predictor of college performance, a stronger predictor than high school GPA, and the SAT remains a significant predictor after adjusting for income and racial/ethnic group.

The empirical evidence clearly shows that the SAT is not biased, such that SAT scores are a valid predictor of college success regardless of your demographic background. In fact, the strength of the relationship between SAT scores and college performance appears to be getting stronger in recent years, whereas the strength of the relationship between high school GPA and college performance is getting weaker.

Here are two main conclusions from the report:

• “Overall, both grades and admissions test scores are moderate predictors of college GPA at UC. The predictive power of test scores has gone up, and the predictive power of high school grades has gone down, since the 2010… study of this issue. At present, test scores are a slightly better predictor of freshman grades than high school grades are. Both grades and scores are stronger predictors of early outcomes (freshman retention and GPA) than of longer-term outcomes (eventual graduation and graduation GPA).”
• “Test scores contribute significant predictive power across all income levels, ethnic groups, across both first-generation and non-first-generation students, and across all campuses and majors.”

Based on these results (and more), the special task force recommended that the UC continue to use the SAT in the admissions process.

But then, on May 21, 2020, the University of California Regents released a statement. They announced their decision to drop the SAT requirement for all applicants to all UC schools. I was shocked. Drop the SAT? Why? The task force report, consistent with previous research, showed that the SAT is not biased and predicts college performance. The task force recommendation was to continue with the SAT. The recommendation was supported by strong empirical evidence and came from a group of 20 highly respected UC faculty members who spent a great deal of time and effort on the report.

The task force report:

“The resulting report and recommendations, grounded in evidence-based research and UC values, reflects the high quality of work one expects of UC faculty… The Subcommittee conducted original analyses of new data, endured long and intense conversations about the analyses and recommendations—including over the winter holiday season, on weekends, while on family vacation, and after hours—and relentlessly kept at the task of writing...”

Why did the UC Regents reject empirical evidence and disregard the recommendation of their own faculty?

Here is the official statement:

“In this action item, the President of the University recommends that the Regents approve a suspension of the current standardized test (ACT/SAT) requirement for undergraduate admissions until 2024 to allow the University to modify or create a new test that better aligns with the content UC expects applicants to have learned and with UC’s values.”

According to the statement, there are two reasons for the UC decision to drop the SAT: (1) The content of the SAT does not align with the content UC expects applicants to have learned. (2) The SAT does not align with the UC’s values.

The first reason is understandable. The content of the SAT is mainly verbal (reading comprehension and writing) and mathematical (quantitative reasoning). It is reasonable for the UC to prefer a test that covers more and/or different content.

But as for the second reason, how does the SAT fail to align with UC’s values? Which values?

Based on comments from UC administrators, the issue is clearly diversity, and the problem is under-representation. African-Americans comprise about 6 percent of California’s population between ages 18 and 24. By comparison, African-American students comprise only 2 percent of the UC student population. There has been little progress over the years. In 2010, the percentages were essentially the same (7 percent and 3 percent). The UC expressed concern that the SAT is not helping to solve the problem of under-representation. But does the SAT (and ACT) actually contribute to the problem?

To answer that question I need to discuss some more results from the task force report. I get into the details of that analysis in the second half of the column: "Part 2: The Reason"