I love having objective, clear-cut descriptions of what my students are supposed to learn, and nice, objective, clear-cut assessments that I can use to measure them. Unfortunately, that’s never going to happen.
These days, I content myself with having a rich and varied set of tasks for my students to do, the sum total of which are the evidence of my students’ mastery of the course content.
My university has these lovely course objectives that get written up for each of the courses I teach. Problem is I could easily tie almost anything I decide to do to those objectives, as could almost anyone else, which kind of makes them meaningless.
What should we have instead? I don’t know yet, but I know tests and exams aren’t the answer any more than fuzzy, impossible to quantify objectives.
I teach at a big state university, and I often receive emails from software companies offering to help me do a basic part of my job: figuring out what my students have learned.
If you thought this task required only low-tech materials like a pile of final exams and a red pen, you’re stuck in the 20th century. In 2018, more and more university administrators want campuswide, quantifiable data that reveal what skills students are learning. Their desire has fed a bureaucratic behemoth known as learning outcomes assessment. This elaborate, expensive, supposedly data-driven analysis seeks to translate the subtleties of the classroom into PowerPoint slides packed with statistics — in the hope of deflecting the charge that students pay too much for degrees that mean too little.
It’s true that old-fashioned course grades, skewed by grade inflation and inconsistency among schools and disciplines, can’t tell us everything about what students have learned. But the ballooning assessment industry — including the tech companies and consulting firms that profit from assessment — is a symptom of higher education’s crisis, not a solution to it. It preys especially on less prestigious schools and contributes to the system’s deepening divide into a narrow tier of elite institutions primarily serving the rich and a vast landscape of glorified trade schools for everyone else.
Without thoughtful reconsideration, learning assessment will continue to devour a lot of money for meager results. The movement’s focus on quantifying classroom experience makes it easy to shift blame for student failure wholly onto universities, ignoring deeper socio-economic reasons that cause many students to struggle with college-level work. Worse, when the effort to reduce learning to a list of job-ready skills goes too far, it misses the point of a university education.
The regional accrediting agencies that certify the quality of education an institution provides — and its fitness to receive federal student financial aid — now require some form of student learning assessment. That means most American colleges and universities have to do it. According to a recent survey, schools deploy an average of four methods for evaluating learning, which include testing software and rubrics to standardize examinations, e-portfolio platforms to display student projects, surveys and other tools.
No intellectual characteristic is too ineffable for assessment. Some schools use lengthy surveys like the California Critical Thinking Disposition Inventory, which claims to test for qualities like “truthseeking” and “analyticity.” The Global Perspective Inventory, administered and sold by Iowa State University, asks students to rate their agreement with statements like “I do not feel threatened emotionally when presented with multiple perspectives” and scores them on metrics like the “intrapersonal affect scale.”
Surveys can’t tell you everything. So universities assemble committees of faculty members, arm them with rubrics and assign them piles of student essays culled from across the school (often called “student products,” as if they are tubes of undergraduate Soylent Green). Assessment has invaded the classroom, too: On many campuses, professors must include a list of skills-based “learning outcomes” on every syllabus and assess them throughout the semester.
All this assessing requires a lot of labor, time and cash. Yet even its proponents have struggled to produce much evidence — beyond occasional anecdotes — that it improves student learning. “I think assessment practices are ripe for re-examining,” said David Eubanks, assistant vice president for assessment and institutional effectiveness at Furman University in Greenville, S.C., who has worked in assessment for years and now speaks out about its problems. “It has forced academic departments to use data that’s not very good,” he added. “And the process of getting this data that’s not very good can be very painful.”
The push to quantify undergraduate learning is about a century old, but the movement really took off in the 1980s. The assessment boom coincided — not, I think, by accident — with the decision of state legislatures all over the country to reduce spending on public universities and other social services. That divestment continued, moving more of the cost of higher education onto students. (These students are often graduates of underfunded high schools that can’t prepare them for college in the first place.) It was politically convenient to hold universities accountable for all this, rather than to scrutinize neoliberal austerity measures.
In 2006, the Commission on the Future of Higher Education, convened by Margaret Spellings, the secretary of education at the time, issued a scathing critique of universities. “Employers report repeatedly that many new graduates they hire are not prepared to work, lacking the critical thinking, writing and problem-solving skills needed in today’s workplaces,” the commission’s report complained.
Educators scrambled to ensure that students graduate with these skills — and to prove it with data. The obsession with testing that dominates primary education invaded universities, bringing with it a large support staff. Here is the first irony of learning assessment: Faced with outrage over the high cost of higher education, universities responded by encouraging expensive administrative bloat.
Many of the professionals who work in learning assessment are former faculty members who care deeply about access to quality education. Pat Hutchings, a senior scholar at the National Institute for Learning Outcomes Assessment (and former English professor), told me: “Good assessment begins with real, genuine questions that educators have about their students, and right now for many educators those are questions about equity. We’re doing pretty well with 18- to 22-year-olds from upper-middle-class families, but what about — well, fill in the blank.”
It seems that the pressure to assess student learning outcomes has grown most quickly at poorly funded regional universities that have absorbed a large proportion of financially disadvantaged students, where profound deficits in preparation and resources hamper achievement. Research indicates that the more selective a university, the less likely it is to embrace assessment. Learning outcomes assessment has become one way to answer the question, “If you get unprepared students in your class and they don’t do well, how does that get explained?” Mr. Eubanks at Furman University told me.
When Erik Gilbert, a professor of history at Arkansas State University, reached the end of his World Civilization course last fall, he dutifully imposed the required assessment: an extra question on the final exam that asked students to read a document about Samurai culture and answer questions using knowledge of Japanese history. Yet his course focused on “cross-cultural connections, trade, travel, empire, migration and bigger-scale questions, rather than area studies,” Mr. Gilbert told me. His students had not studied Japanese domestic history. “We do it this way because it satisfies what the assessment office wants, not because it addresses concerns that we as a department have.”
Mr. Gilbert became an outspoken assessment skeptic after years of watching the process fail to capture what happens in his classes — and seeing it miss the real reasons students struggle. “Maybe all your students have full-time jobs, but that’s something you can’t fix, even though that’s really the core problem,” he said. “Instead, you’re expected to find some small problem, like students don’t understand historical chronology, so you might add a reading to address that. You’re supposed to make something up every semester, then write up a narrative” explaining your solution to administrators.
Here is the second irony: Learning assessment has not spurred discussion of the deep structural problems that send so many students to college unprepared to succeed. Instead, it lets politicians and accreditors ignore these problems as long as bureaucratic mechanisms appear to be holding someone — usually a professor — accountable for student performance.
All professors could benefit from serious conversations about what is and is not working in their classes. But instead they end up preoccupied with feeding the bureaucratic beast. “It’s a bit like the old Soviet Union. You speak two languages,” said Frank Furedi, an emeritus professor of sociology at the University of Kent in Britain, which has a booming assessment culture. “You do a performance for the sake of the auditors, but in reality, you carry on.”
Yet bureaucratic jargon subtly shapes the expectations of students and teachers alike. On the first day of class, my colleagues and I — especially in the humanities, where professors are perpetually anxious about falling enrollment — find ourselves rattling off the skills our courses offer (“Critical thinking! Clear writing!”), hyping our products like Apple Store clerks.
I teach intellectual history. Of course that includes skills: learning to read a historical source, interpret evidence and build an argument. But cultivating historical consciousness is more than that: It means helping students immerse themselves in a body of knowledge, question assumptions about memory and orient themselves toward current events in a new way.
If we describe college courses as mainly delivery mechanisms for skills to please a future employer, if we imply that history, literature and linguistics are more or less interchangeable “content” that convey the same mental tools, we oversimplify the intellectual complexity that makes a university education worthwhile in the first place. We end up using the language of the capitalist marketplace and speak to our students as customers rather than fellow thinkers. They deserve better.
“When kids come from backgrounds where they’re the first in their families to go to college, we have to take them seriously, and not flatter them and give them third-rate ideas,” Mr. Furedi told me. “They need to be challenged and inspired by the idea of our disciplines.” Assessment culture is dumbing down universities, he said: “One of the horrible things is that many universities think that giving access to nontraditional students means turning a university into a high school. That’s not giving them access to higher education.”
Here is the third irony: The value of universities to a capitalist society depends on their ability to resist capitalism, to carve out space for intellectual endeavors that don’t have obvious metrics or market value.
Consider that holy grail of learning outcomes, critical thinking — what the philosopher John Dewey called the ability “to maintain the state of doubt and to carry on systematic and protracted inquiry.” Teaching it is not a cheap or efficient process. It does not come from trying to educate the most students at the lowest possible cost or from emphasizing short, quantifiable, standardized assignments at the expense of meandering, creative and difficult investigation.
Molly Worthen is the author, most recently, of “Apostles of Reason: The Crisis of Authority in American Evangelicalism,” an assistant professor of history at the University of North Carolina, Chapel Hill, and a contributing opinion writer.
I invite you to join me on Twitter, @MollyWorthen.
Follow The New York Times Opinion section on Facebook and Twitter (@NYTopinion), and sign up for the Opinion Today newsletter.
