Transforming Assurance of Learning for Lasting Impact

Sponsored Content

Assurance of learning (AoL) has been an integral part of the AACSB accreditation process since 2003. This form of course-embedded assessment provides AACSB member schools with a defined process, driven by rubrics at the course or module level. AoL helps schools map the achievement of program-level learning goals across bachelor’s, master’s, and doctoral programs.

AoL, as defined in AACSB’s 2020 business accreditation standards, “refers to the systematic processes and assessment plans that collectively demonstrate that learners achieve learning competencies for the programs in which they participate that are within the scope of the school’s accreditation. AACSB accreditation is concerned with broad, program-level, focused competency goals for each degree program, rather than detailed competency goals by course or topic.” Essentially, AoL measures a business school’s commitment to learners that, at the end of their programs, they will be able to demonstrate proficiency over a set of competencies, or goals, defined at the program level.

Globally, many hundreds of different AoL systems exist across schools accredited by AACSB and those in the accreditation process that already have AoL systems in place. While all of these systems adhere to the same guidelines that are now explained in great detail in Standard 5 of the 2020 standards, AoL administration can vary in different regions.

When the 2003 standards were introduced, one of the authors had the great pleasure of inviting Kathryn Martell, an early AoL expert, to work with the entire Coles College of Business faculty on the strategy and execution of assurance of learning. Years later, Martell, who is now at Central Washington University and an established global AoL expert, pointed out that in some parts of the world—Europe, Mexico, and the Middle East, for example—it is common to have staff devoted to accreditation and AoL. However, these staff, who collect and analyze data for faculty review, are much less common in the U.S.

Despite some regional differences in how AoL is managed, all schools face a new need to revisit their AoL processes, driven by impacts from the 2020 standards as well as the mass digitization of learning and teaching resulting from the COVID-19 pandemic.

The Evolution of AoL

With the 2020 standards, schools are challenged to review their AoL processes, identifying a better incorporation of indirect measures. Indirect measures of learning refer to evidence attained from third-party input that is not measured at the individual learner level. It is that third-party perspective that can add different and valuable insights, complementing direct measures of learning.

For example, at Shippensburg University in Pennsylvania, Dean John Kooti used an undergraduate program assessment, provided by a third party and including nationally based norms, to demonstrate indirect measures of content area knowledge, ethical reasoning, leadership, and global perspectives. Northern Illinois University, as another example, incorporates input from graduation and alumni surveys, as well as from both internal and external advisory boards to improve curriculum.

---

With the 2020 standards, schools are challenged to review their AoL processes, identifying a better incorporation of indirect measures.

---

Some other exceptional examples of indirect measures, gathered from both the University of Richmond Robins School of Business in Virginia and the St. Louis University Chaifetz School of Business in Missouri, include the following:

• Survey of students completing internships
• Survey of internship employers
• Survey of full-time employers of alumni
• Alumni surveys, one year, three years, five years post-employment
• Graduation surveys
• Peer benchmarking

Thunderbird School of Global Management at Arizona State University in Phoenix has developed a Global Mindset Inventory that is used by more than 30 business schools globally. Indeed, Dean Sanjeev Khagram reports that, following a recent aggregation of global data, Thunderbird adjusted its curriculum to add a global consulting project that would enhance social and psychological capital. This is truly “closing the loop”—showing that the curriculum was improved by the AoL process.

The What and Where of Assessment

In many instances globally, courses and modules were rushed online during the pandemic without any changes to the instructional approaches or methods. Now that schools have had some time to properly establish online courses, they are starting to adapt their didactics to fully leverage the possibilities of online learning. We have seen extraordinary examples of virtual teams working across continents, students completing virtual internships and/or consulting projects, and a host of other virtually driven initiatives that are replacing historical face-to-face efforts.

As a result of the enormous challenges created by the pandemic, schools are facing new concerns about the time cycle for AoL and assessment. We ask business schools to consider whether they’ve encountered these or other challenges specifically related to the “what and where” of assessment:

• Has your business school made small or large changes to your measurement of learning outcomes? If so, will you modify those measures again as you return to either face-to-face or a blended learning model?
• Is your business school skipping an AoL cycle, hoping things will go back to normal, or will you completely revise your processes?
• Do you assess student presentations the same way as you did pre-pandemic? How about group work or consulting field projects, etc.?

The answers to these questions will impact the extent to which schools may need to update their documented AoL processes and re-evaluate lessons learned in their current assessment cycles. Schools will need to carefully analyze measurement results against their current situation to decide whether actions for improvement are circumstantial or structural in nature.

Digitizing the AoL Process

Traditionally, schools have used paper-based rubrics to assess learner success embedded in their AoL processes. The notion of paper-based evaluation is deeply rooted in higher education and is hard to overcome. Take course evaluations as an example. We know of schools that have seen course evaluation completion rates drop from around 80 percent of paper-based evaluations to as low as 20 percent when moving to online evaluations.

Digitally submitted student papers often get printed and graded with pen and paper. Despite advances in technology, the habit of completing certain tasks on paper rather than on the screen is difficult to break, even as it typically means additional work, such as the printing itself as well as eventually digitizing the hand-written results.

The pandemic and the resulting need to teach and learn remotely have forced many business educators to rethink old habits. In the past, you could walk down campus to drop-off a heavy box full of completed rubrics. Of course, nothing stops faculty from shipping these rubric anywhere, but that would naturally not be the most efficient and effective option.

So how do you break the chains of habit? How do you convince faculty to embrace digitization beyond the means to an end during a pandemic? Why is it that, for many traditional universities, digitizing teaching and assessment often was a despised necessity?

---

To overcome potential resistance, try focusing less on what needs to be done and more on how it will benefit faculty in the end.

---

Any change process, such digitizing, requires effort and resources up front. While much of the workload can be handled by project teams, such projects will not be successful without faculty and staff involvement, as their work will be highly affected by the changes. To overcome potential resistance, try focusing less on what needs to be done and more on how it will benefit faculty in the end.

A well-executed digitized AoL system bridges the gap between various stakeholders: faculty, students, assessment offices, AoL officers, and others. Such a system seemingly integrates with existing learning management systems to avoid data silos. If you only replace an analog step in the process with a digital step, you do not gain anything. Rather, a fully digitized system should allow you to streamline your overall processes, reducing duplicate tasks and increasing accessibility of data in terms of where and by whom.

AoL: Making It Work Post-Pandemic

Today, an increased number of options exist for schools to design and execute their AoL processes. Recent shifts in the standards and in online teaching and learning present a compelling need for business school faculty and staff to review and adapt their current processes.

As we emerge from the pandemic, now is the time to fully commit to digitizing assurance of learning; doing so will help you avoid unnecessary complexities, reduce the burden on classroom instructors, track data gathering and collection efforts, support statistical sampling of students, and assist in mapping the AoL process. The insights gained through digitization will guide you to opportunities to “close the loop” and achieve an enriched curriculum for learner success.