Improving Learner Outcomes With Academic Leveling

Sponsored Content

• Many students seeking entry into an MBA program are non-business majors, presenting challenges with foundational knowledge for business schools.
• In a Peregrine study of 17 client institutions, learners who completed a module-based leveling program at the start of their graduate programs had higher retained knowledge compared to learners who did not complete the program.
• Regardless of undergraduate major, academic leveling modules help reinforce foundational concepts essential to learning within graduate business programs.

 
An MBA is one of the most versatile graduate degrees a learner can pursue. Regardless of industry, an MBA prepares students to achieve their entrepreneurial dreams, change careers, or move into leadership positions. It is no surprise, then, that non-business majors make up a large percentage of students who seek MBA degrees. According to a 2017 Graduate Management Admission Council report of over 11,000 prospective students surveyed, more than half of all students seeking entry into an MBA program are non-business majors. This high proportion creates an issue for business schools, as non-business majors may lack foundational business knowledge, which can negatively impact their success. The use of module-based online content in graduate business programs has proven to effectively help students acquire—and retain—this knowledge.

Academic leveling refers to a process in which students become oriented, or leveled, to a field of study by reviewing the fundamentals of that subject. The use of leveling modules at the beginning of or prior to a student’s graduate program can provide that foundational knowledge. Further, a modular approach to providing online leveling content allows institutions to choose the topic areas most vital to success in their programs and deliver them in chunks that promote greater integration and transfer of knowledge.

Modules are paired with a pre-test and a post-test to promote learning through assessment and help institutions effectively measure learner competency within business subjects. The online, self-guided modules provide flexibility to institutions, as their students can complete the modules asynchronously either before admissions, in an introductory course, or in conjunction with other coursework required for their programs.

Research into cognitive science shows that learners can better understand new ideas, retain information, solve problems, and transfer knowledge to novel situations when they have a solid foundation of facts committed to long-term memory. Deans for Impact has identified key cognitive principles that guide curriculum development to harness existing research on the ways people learn. Consider the following principles:

• How students understand new ideas: Students learn new ideas by referencing ideas they already know. Knowledge builds upon itself, so a well-sequenced curriculum must ensure that students have the prior knowledge to master new concepts. 
• How students learn and retain new information: When instructors space activities to reinforce concepts over time, they help learners retain information for the long term. 
• How students solve problems: Students have a limited capacity of working memory, which is needed to solve problems. By committing specific key facts to long-term memory beforehand, they free up working memory for problem-solving and can access facts, knowledge, and skills from long-term memory when required.
• How students transfer learning to new situations: The ability to transfer knowledge to a novel problem requires knowledge of the problem's context and structure. When educators provide sufficient background knowledge to students, the students' ability to transfer knowledge increases.

The cognitive principles support use of academic leveling as a learner matriculates into a graduate program. By introducing key concepts to students at the beginning of a program and later reinforcing them during study, educators help ensure that students efficiently retrieve the concepts when they need them for problem-solving. Over the years, higher education institutions have used academic leveling modules (ALMs) in combination with Peregrine's programmatic assessment. Consequently, Peregrine has access to a wealth of data from approximately 500 client schools worldwide that we have analyzed to discover trends in retained knowledge.

Through such an analysis, we found that the short-term impact on developing foundational knowledge was outstanding. Comparing pre-test and post-test data over four academic years (2016 to 2020) shows that learners experienced an average percentage change (knowledge gain) of 81 percent based on the differences between the pre-test and post-test module scores.

The following figure shows the pooled pre-test and post-test scores for 15 modules used for academic leveling between 2016 and 2020 (a 16th module, Business Communications, was added for academic year 2020–21). The comparison of pre-test and post-test scores clearly demonstrated a gain in foundational business knowledge after a learner completed the module’s instructional content. But the question remained, how does leveling affect retained knowledge in the long term?

Figure 1. Pooled Pre-Test and Post-Test Scores for Academic Years 2016­–20

Pooled pre-test and post-test scores for the 15 academic leveling modules for the 2016–17, 2017–18, 2018–19, and 2019–­20 academic years (July 1­–June 30).

To answer this question, Peregrine investigated data from 17 client institutions over a six-year period (2014 to 2020) using data from the Business Administration Assessment Outbound Exam. To meet sampling criteria, schools must have had at least two years of outbound exam data before academic leveling and at least one year of outbound exam data after academic leveling. Schools also needed to have a minimum of 30 completed exams in a given academic year to be included in the population analysis.

The pre-leveling sample was defined as learners within the sample who completed an outbound exam at the end of their degree program and did not complete the leveling program at the start of their program. This sample included learners primarily in MBA programs, with some in MS or MA programs. The resulting annual sample size for each academic year ranged from 604 to 2,293 exams for the analysis population.

In total for the six-year analysis period, the pre-leveling outbound exam score was 52.40 percent (n=5,864 outbound exams) and the post-leveling outbound exam score was 56.41 percent (n=8,589 outbound exams), which represents a difference of 4.01 percent and a 7.64 percentage change (improvement) in the exam scores.

Figure 2. Pre-Leveling and Post-Leveling Outbound Exam Scores of Master’s-Degree Learners

By outbound exam topic, the scores of learners who did not complete the leveling program (n=5,864 exams, average score: 52.40 percent) and learners who did complete the leveling program (n=8,589 exams, average score: 56.41 percent). 

Overall, learners who completed the module-based leveling program at the start of their graduate programs had higher retained knowledge compared to learners who did not complete the program.

One school, featured as a case study in the whitepaper “Understanding Academic Leveling: Case Studies and Data Trends,” found that using academic leveling modules for all incoming graduate students, regardless of undergraduate major, also had a positive influence on students’ retained knowledge. The institution not only eliminated the need to track and monitor which incoming students would take the leveling modules but also used the modules in place of previous prerequisites. Since the institution used Peregrine’s assessment prior to adopting this change, they were able to study the longitudinal effect of required leveling modules for all learners.

The outbound exam results from July 2017 through December 2020 (n=694) were summarized by annual time periods and separated into two groups: outbound exam results from students graduating from the program who did not complete the leveling modules at the start of their program (n=588), and outbound exam results from students who did complete the leveling modules at the start of their program (n=106). The first three time periods included students who did not complete the leveling program (pre-leveling), and the last two time periods included students who did complete the leveling program (post-leveling). The data were evaluated using regression analysis.

Learners’ outbound exam scores consistently improved for those who completed the leveling modules at the start of their master’s programs: R2=0.9412. Post-leveling outbound exam scores (n=106) were significantly higher than pre-leveling outbound exam scores (n=588). Therefore, regardless of undergraduate major, academic leveling modules help reinforce foundational concepts essential to learning within graduate business programs.

Figure 3. Master’s Outbound Exam: Total Exam Score

Periods 1, 2, and 3 are pre-leveling; periods 4, 5, and 6 are post-leveling.

Outbound exam total scores from July 1, 2017, through December 31, 2020, plotted by time period. Post-leveling outbound exam scores (n=106) were significantly higher than the pre-leveling outbound exam scores (n=588). 

Our analysis of data from programmatic assessment and academic leveling shows that modular-based activities at the beginning of a graduate program introduce or reinforce foundational concepts and have significant, positive impacts on both learning and retaining knowledge. Although the original purpose of the modules was to level learners from non-business backgrounds or adult learners who have been out of school for several years, we have found benefits for students regardless of academic major.

Download the full whitepaper, “Understanding Academic Leveling: Case Studies and Data Trends,” to get a thorough look at the data analyzed and read further into institution-specific case studies.