The Association for the Assessment of Learning in Higher Education (AALHE) held its annual conference from June 3 to 8 in Milwaukee, Wisconsin. One of our Digital Pedagogy and Scholarship staff members – Leslie Harris – attended the conference, and he provides this overview of conference highlights and in particular of the three plenary sessions that framed the conference.
The opening keynote address was delivered by Thomas J. Chapel, Chief Evaluation Officer for the Centers for Disease Control and Prevention in Washington, D.C. His talk focused on "Potholes on the Road to Good Monitoring and Evaluation," setting an overarching theme for the conference by focusing on assessment and evaluation as part of a cycle of continuous quality improvement. He used the personal example of coaching a soccer team with a losing record. If we focus on just one overarching final goal (winning games), we miss a series of important interim goals that we can assess on the way to that ultimate outcome. For example, are the (young) players passing the ball to their teammates (and not to the other team)? Are the players staying in position? Are they keeping the ball in the opposing half of the field as much as possible? By focusing on these "intermediate outcomes" – what skills we hope our students/players will learn as a result of taking our courses or participating in our programs (or in his analogy, attending practices and playing in games) – we can be involved in a process of assessment and continuous improvement that will help us reach our goals. Mr. Chapel introduced another key theme of the conference by repeating a famous quotation often attributed to Albert Einstein but actually from Bruce Cameron:
Not everything that can be counted counts, and not everything that counts can be counted.
Chapel had several points in mind when citing that quotation, ideas that were stressed in other sessions as well:
- We don’t want to evaluate only what is easy to measure.
- What we measure reflects what we value.
- We want to try to assess the most important questions for our stakeholders.
The second plenary session, "Leading with Assessment: Ensuring That What Counts for Us Matters for Our Students" reinforced that theme of paying attention to what matters most to students. Charles Blaich, director of the Center of Inquiry at Wabash College and the Higher Education Data Sharing Consortium (HEDS), and Kathleen Wise, associate director, reminded the audience of misleading uses of statistics by marketing campaigns, even campaigns from our own universities. They cited one university (for example) that claims that 93% of their graduates are “set” – in other words, in some kind of job – after graduation, even if that vague category of “set” actually includes part-time jobs that have no relationship to the graduate’s major or life goals. They stated that Assessment professionals are in a unique position to:
- lead honest inquiry about the extent to which our institutions help our students achieve the goals we assert in our missions,
- discover other ways our students do and do not benefit from attending our institutions, and
- develop an authentic narrative about what our institutions do, and don’t do, well.
The final plenary session delivered by Jeremy Penn, Director of Student Affairs Assessment at North Dakota State University, focused on "Reframing Assessment as Meaning Making." Jeremy asserted that “Assessment is about making meaning from this complex, messy, difficult thing called learning,” and that program assessment can help students make "sense of their own learning transformation" through their years at college. In other words, students may wonder "What did I learn in this course? What did I gain from this co-curricular experience? How have I changed as a result of my years in college?" By assessing and reflecting on their own learning, students make meaning out of their college experience. He acknowledged that "Our programs do struggle to understand their impact on student learning and development," but he emphasized that "Assessment is a critical component of supporting learning and student success."