{"id":6850,"date":"2018-02-24T14:59:10","date_gmt":"2018-02-24T21:59:10","guid":{"rendered":"http:\/\/minkhollow.ca\/beckerblog\/?p=6850"},"modified":"2018-02-24T14:59:10","modified_gmt":"2018-02-24T21:59:10","slug":"worth-sharing-the-misguided-drive-to-measure-learning-outcomes-nyt","status":"publish","type":"post","link":"https:\/\/minkhollow.ca\/beckerblog\/2018\/02\/24\/worth-sharing-the-misguided-drive-to-measure-learning-outcomes-nyt\/","title":{"rendered":"Worth Sharing: The Misguided Drive to Measure \u2018Learning Outcomes\u2019 NYT"},"content":{"rendered":"Approximate Reading Time: <\/span> 7<\/span> minutes<\/span><\/span>

\"\"<\/a>Colleges should stop trivializing the transmission of knowledge.<\/p><\/blockquote>\n

Source: Opinion | The Misguided Drive to Measure \u2018Learning Outcomes\u2019<\/a><\/em><\/p>\n

I love having objective, clear-cut descriptions of what my students are supposed to learn, and nice, objective, clear-cut assessments that I can use to measure them. Unfortunately,\u00a0that’s never going to happen<\/strong><\/em>.<\/p>\n

These days, I content myself with having a rich and varied set of tasks for my students to do, the sum total of which are the evidence of my students’ mastery of the course content.<\/p>\n

My university has these lovely course objectives that get written up for each of the courses I teach. Problem is I could easily tie almost anything I decide to do to those objectives, as could almost anyone else, which kind of makes them meaningless.<\/p>\n

 <\/p>\n

What should we have instead? I don’t know yet, but I know tests and exams aren’t the answer any more than fuzzy, impossible to quantify objectives.<\/p>\n

I teach at a big state university, and I often receive emails from software companies offering to help me do a basic part of my job: figuring out what my students have learned.<\/p>\n

If you thought this task required only low-tech materials like a pile of final exams and a red pen, you\u2019re stuck in the 20th century. In 2018, more and more university administrators want campuswide, quantifiable data that reveal what skills students are learning. Their desire has fed a bureaucratic behemoth known as learning outcomes assessment. This elaborate, expensive, supposedly data-driven analysis seeks to translate the subtleties of the classroom into PowerPoint slides packed with statistics \u2014 in the hope of deflecting the charge that students pay too much for degrees that mean too little.<\/p>\n

It\u2019s true that old-fashioned course grades, skewed by grade inflation and inconsistency among schools and disciplines, can\u2019t tell us everything about what students have learned. But the ballooning assessment industry \u2014 including the tech companies and consulting firms that profit from assessment \u2014 is a symptom of higher education\u2019s crisis, not a solution to it. It preys especially on less prestigious schools and contributes to the system\u2019s deepening divide into a narrow tier of elite institutions primarily serving the rich and a vast landscape of glorified trade schools for everyone else.<\/p>\n

Without thoughtful reconsideration, learning assessment will continue to devour a lot of money for meager results. The movement\u2019s focus on quantifying classroom experience makes it easy to shift blame for student failure wholly onto universities, ignoring deeper socio-economic reasons that cause many students to struggle with college-level work. Worse, when the effort to reduce learning to a list of job-ready skills goes too far, it misses the point of a university education.<\/p>\n

The regional accrediting agencies that certify the quality of education an institution provides \u2014 and its fitness to receive federal student financial aid \u2014 now require some form of student learning assessment. That means most American colleges and universities have to do it. According to a recent survey, schools deploy an average of four methods for evaluating learning, which include testing software and rubrics to standardize examinations, e-portfolio platforms to display student projects, surveys and other tools.<\/p>\n

No intellectual characteristic is too ineffable for assessment. Some schools use lengthy surveys like the California Critical Thinking Disposition Inventory, which claims to test for qualities like \u201ctruthseeking\u201d and \u201canalyticity.\u201d The Global Perspective Inventory, administered and sold by Iowa State University, asks students to rate their agreement with statements like \u201cI do not feel threatened emotionally when presented with multiple perspectives\u201d and scores them on metrics like the \u201cintrapersonal affect scale.\u201d<\/p>\n

Surveys can\u2019t tell you everything. So universities assemble committees of faculty members, arm them with rubrics and assign them piles of student essays culled from across the school (often called \u201cstudent products,\u201d as if they are tubes of undergraduate Soylent Green). Assessment has invaded the classroom, too: On many campuses, professors must include a list of skills-based \u201clearning outcomes\u201d on every syllabus and assess them throughout the semester.<\/p>\n

All this assessing requires a lot of labor, time and cash. Yet even its proponents have struggled to produce much evidence \u2014 beyond occasional anecdotes \u2014 that it improves student learning. \u201cI think assessment practices are ripe for re-examining,\u201d said David Eubanks, assistant vice president for assessment and institutional effectiveness at Furman University in Greenville, S.C., who has worked in assessment for years and now speaks out about its problems. \u201cIt has forced academic departments to use data that\u2019s not very good,\u201d he added. \u201cAnd the process of getting this data that\u2019s not very good can be very painful.\u201d<\/p>\n

The push to quantify undergraduate learning is about a century old, but the movement really took off in the 1980s. The assessment boom coincided \u2014 not, I think, by accident \u2014 with the decision of state legislatures all over the country to reduce spending on public universities and other social services. That divestment continued, moving more of the cost of higher education onto students. (These students are often graduates of underfunded high schools that can\u2019t prepare them for college in the first place.) It was politically convenient to hold universities accountable for all this, rather than to scrutinize neoliberal austerity measures.<\/p>\n

In 2006, the Commission on the Future of Higher Education, convened by Margaret Spellings, the secretary of education at the time, issued a scathing critique of universities. \u201cEmployers report repeatedly that many new graduates they hire are not prepared to work, lacking the critical thinking, writing and problem-solving skills needed in today\u2019s workplaces,\u201d the commission\u2019s report complained.<\/p>\n

Educators scrambled to ensure that students graduate with these skills \u2014 and to prove it with data. The obsession with testing that dominates primary education invaded universities, bringing with it a large support staff. Here is the first irony of learning assessment: Faced with outrage over the high cost of higher education, universities responded by encouraging expensive administrative bloat.<\/p>\n

Many of the professionals who work in learning assessment are former faculty members who care deeply about access to quality education. Pat Hutchings, a senior scholar at the National Institute for Learning Outcomes Assessment (and former English professor), told me: \u201cGood assessment begins with real, genuine questions that educators have about their students, and right now for many educators those are questions about equity. We\u2019re doing pretty well with 18- to 22-year-olds from upper-middle-class families, but what about \u2014 well, fill in the blank.\u201d<\/p>\n

It seems that the pressure to assess student learning outcomes has grown most quickly at poorly funded regional universities that have absorbed a large proportion of financially disadvantaged students, where profound deficits in preparation and resources hamper achievement. Research indicates that the more selective a university, the less likely it is to embrace assessment. Learning outcomes assessment has become one way to answer the question, \u201cIf you get unprepared students in your class and they don\u2019t do well, how does that get explained?\u201d Mr. Eubanks at Furman University told me.<\/p>\n

When Erik Gilbert, a professor of history at Arkansas State University, reached the end of his World Civilization course last fall, he dutifully imposed the required assessment: an extra question on the final exam that asked students to read a document about Samurai culture and answer questions using knowledge of Japanese history. Yet his course focused on \u201ccross-cultural connections, trade, travel, empire, migration and bigger-scale questions, rather than area studies,\u201d Mr. Gilbert told me. His students had not studied Japanese domestic history. \u201cWe do it this way because it satisfies what the assessment office wants, not because it addresses concerns that we as a department have.\u201d<\/p>\n

Mr. Gilbert became an outspoken assessment skeptic after years of watching the process fail to capture what happens in his classes \u2014 and seeing it miss the real reasons students struggle. \u201cMaybe all your students have full-time jobs, but that\u2019s something you can\u2019t fix, even though that\u2019s really the core problem,\u201d he said. \u201cInstead, you\u2019re expected to find some small problem, like students don\u2019t understand historical chronology, so you might add a reading to address that. You\u2019re supposed to make something up every semester, then write up a narrative\u201d explaining your solution to administrators.<\/p>\n

Here is the second irony: Learning assessment has not spurred discussion of the deep structural problems that send so many students to college unprepared to succeed. Instead, it lets politicians and accreditors ignore these problems as long as bureaucratic mechanisms appear to be holding someone \u2014 usually a professor \u2014 accountable for student performance.<\/p>\n

All professors could benefit from serious conversations about what is and is not working in their classes. But instead they end up preoccupied with feeding the bureaucratic beast. \u201cIt\u2019s a bit like the old Soviet Union. You speak two languages,\u201d said Frank Furedi, an emeritus professor of sociology at the University of Kent in Britain, which has a booming assessment culture. \u201cYou do a performance for the sake of the auditors, but in reality, you carry on.\u201d<\/p>\n

Yet bureaucratic jargon subtly shapes the expectations of students and teachers alike. On the first day of class, my colleagues and I \u2014 especially in the humanities, where professors are perpetually anxious about falling enrollment \u2014 find ourselves rattling off the skills our courses offer (\u201cCritical thinking! Clear writing!\u201d), hyping our products like Apple Store clerks.<\/p>\n

I teach intellectual history. Of course that includes skills: learning to read a historical source, interpret evidence and build an argument. But cultivating historical consciousness is more than that: It means helping students immerse themselves in a body of knowledge, question assumptions about memory and orient themselves toward current events in a new way.<\/p>\n

If we describe college courses as mainly delivery mechanisms for skills to please a future employer, if we imply that history, literature and linguistics are more or less interchangeable \u201ccontent\u201d that convey the same mental tools, we oversimplify the intellectual complexity that makes a university education worthwhile in the first place. We end up using the language of the capitalist marketplace and speak to our students as customers rather than fellow thinkers. They deserve better.<\/p>\n

\u201cWhen kids come from backgrounds where they\u2019re the first in their families to go to college, we have to take them seriously, and not flatter them and give them third-rate ideas,\u201d Mr. Furedi told me. \u201cThey need to be challenged and inspired by the idea of our disciplines.\u201d Assessment culture is dumbing down universities, he said: \u201cOne of the horrible things is that many universities think that giving access to nontraditional students means turning a university into a high school. That\u2019s not giving them access to higher education.\u201d<\/p>\n

Here is the third irony: The value of universities to a capitalist society depends on their ability to resist capitalism, to carve out space for intellectual endeavors that don\u2019t have obvious metrics or market value.<\/p>\n

Consider that holy grail of learning outcomes, critical thinking \u2014 what the philosopher John Dewey called the ability \u201cto maintain the state of doubt and to carry on systematic and protracted inquiry.\u201d Teaching it is not a cheap or efficient process. It does not come from trying to educate the most students at the lowest possible cost or from emphasizing short, quantifiable, standardized assignments at the expense of meandering, creative and difficult investigation.<\/p>\n

\n

Molly Worthen<\/a>\u00a0is the author, most recently, of \u201cApostles of Reason: The Crisis of Authority in American Evangelicalism,\u201d an assistant professor of history at the University of North Carolina, Chapel Hill, and a contributing opinion writer.<\/p>\n<\/div>\n

\n

I invite you to join me on Twitter,\u00a0@MollyWorthen<\/a>.<\/em><\/p>\n

Follow The New York Times Opinion section on\u00a0Facebook<\/a>\u00a0and\u00a0Twitter (@NYTopinion)<\/a>, and sign up for the\u00a0Opinion Today newsletter<\/a>.\u00a0<\/em><\/p>\n<\/div>\n

 <\/p><\/blockquote>\n