Don’t blame SATs for American inequality Jack Buckley et al.'s 'Measuring Success'

It’s essential to understand what kind of student was particularly well-served by the SAT: the kind who is bright and talented but who had failed to live up to their potential in class. These students tended to be the brilliant dreamers; they were the ones in possession of uncommon cognitive skills, but who performed poorly in knowledge-based exams because of bad time management, resistance to the indignities of organised education, or an inability to prioritise school over their own interests. For decades, excellent SAT scores got students into colleges that they wouldn’t ordinarily get into, creating opportunities to find diamonds in the rough who had perhaps never found their footing in school.

The students who the SAT disadvantaged, in contrast, were the grinders — those who were not particularly talented or gifted but who were able to get good grades through ingratiating themselves to teachers, studying maniacally, and always getting whatever points were available for effort. There’s no doubt that society needs both types of student, but the elimination of the SAT benefits only one sort.

There are many fundamental misconceptions that plague discussion of the SATs, and unfortunately, my long experience discussing the test tells me that most people’s ignorance is of the motivated kind. I got my PhD in English, but my focus was on the assessment of student learning and my dissertation was on tests of college learning. And what I learned during that process, and have had reinforced since, is that many people want to maintain myths about such tests out of a sense of resentment and naïve political concerns. This book helpfully dispatches with many common, but bad, arguments.

Consider the claim that the SAT should be abolished because it is not in fact a strong predictor of college performance. Those who make it often throw around correlations between the SAT and freshman year grades that appear to be low. But this is a quirk of how that data is collected. Such analyses can necessarily only consider students who got a high enough SAT score to get into a given college; that is, students who don’t get into college can’t have college grades, and so that whole sector of data is missing from analysis. This is a classic case of range restriction. If we correct for this problem, the SAT’s correlation with freshman grades is robust and consistent. The authors of Measuring Success demonstrate this, before going on to show empirically that SATs predict not just first-year college but a number of other important indicators of life success.

They note, too, that we have known about this range restriction issue for decades, and yet critics of the SAT continue to make this elementary mistake. It speaks to the committed ignorance many people have on this topic. There’s such deep resentment towards high-stakes tests, which cause stress for test-takers before starkly ranking them, that people are willing to look beyond little things like intellectual consistency or responsible evaluation of research.

The GPA is already broadly considered the single-most important criterion for applicants. The crowd that would prefer to give it even more consideration in admissions are at least still sticking with a quantitative indicator; the question is why we would prefer less quantitative data (just GPA) rather than more (GPA plus SAT).

In a nice show of professional respect, Measuring Success reprints a 2009 study that is often invoked by SAT critics. That research found that GPA was a strong predictor of college success, which some take to mean that we can safely discard the SAT, because it adds nothing to our understanding of a student’s potential. But Measuring Success goes on to examine a replication of this study, with newer data, which finds that the predictive accuracy of GPA has fallen, probably due to grade inflation, a problem which does not afflict a test like the SAT, which is normed every year. The SAT’s predictive quality, in contrast, went up.

If nothing else, using the SAT ensures that more information, and more kinds of information, are used in selecting students. But this kind of diversity — diversity of thinking and learning and academic style — is not the kind of diversity that competitive colleges are pursuing. Elite American colleges are already more racially diverse than the country writ large, but the perpetual cry is for more people of colour on campus. This is the source of the most persistent criticisms of the SAT. The broad claim about the SAT is that, since there are race and class disparities in SAT scores — white and Asian pupils score better than Hispanic and black, and rich better than poor — then the test must be discriminatory and should be abolished.

This is a bit like blaming seismographs for earthquakes. The SAT does not create inequality; the SAT reveals inequality. Contemporary progressives believe that black people and poorer people face many hurdles in life, thanks to entrenched inequality. I agree entirely. But if we’ve acknowledged that, how can we blame the SAT when its results are skewed against the poor and racial minorities? All of that inequality has consequences, and one of those consequences is poorer test scores. All educational data is race-and-class stratified, including the gold-standard National Assessment of Educational Progress, various state-specific exams, ancillary data like attendance and discipline, and crucially, GPA. That’s right: the very metric that many critics of the SAT would have us replace the test with shows the same race and class dynamics!

Perhaps this is why test-optional efforts to diversify campuses have often fallen flat. Crucially, Measuring Success presents three studies that consider the question of whether going test-optional results in a more diverse campus — and all three find no evidence to support that claim. For example, the study “The Effect of Going Test-Optional on Diversity and Admissions” finds that going test optional “has no statistically significant effect… on the proportion of underrepresented students of colour enrolling in the institution”. If removing the SAT does not in fact improve minority representation, then what are we doing here, exactly?

Of course, admissions officers have plenty of freedom to decide who they’ll accept, particularly at private institutions. As long as schools aren’t engaged in federally-prohibited race or gender discrimination, the average college can select pretty much whoever they want. The schools dropping the SAT make a big show about diversity and equality, never bothering to note that they and not the test determine who gets the brass ring. Advocates for dropping the SAT ask us to put even more faith in those units of our highly-bureaucratic and self-dealing institutions of higher education. And this, finally, is the most irrational belief of all: the notion that, if we only give the admissions departments of exclusive colleges more power, they will do what’s right and usher in a new age of equity and diversity in higher education.

This amounts to a childish faith in institutions that we have no reason to trust. Colleges and universities design their admissions departments, like any other department, with self-interest in mind. To demonstrate this, simply look at affirmative action at elite universities. These institutions keep their data very close to the vest, but it’s widely believed that schools such as Harvard use their affirmative action seats primarily as money-makers. They are alleged to do so by seeking out rich children of recent immigrants, whose parents (they expect) will be able and willing to donate — which would mean the average Ivy League affirmative action slot isn’t filled with a poor American-born descendant of African slaves, but rather with the upwardly-mobile progeny of a Nigerian surgeon.

This is the reality of diversity at elite colleges. Yet critics of the SAT have dogged faith that the SAT is all that stands in the way of more equitable and diverse admissions. The notion is a mirage. If we’re going to fight discrimination and inequality, we’ll need tools to understand the size of the problem — tools like the SAT. Using them, we can embrace not just racial diversity, but diversity in terms of the strengths we recognise in applicants.