Assessing Critical Thinking in the Digital Era

Sponsored Content

• Technology is poised to revolutionize education. Instead of being disrupted by the new tech, schools should participate in its development.
• Technology can be particularly useful in helping schools assess critical thinking skills, which have become even more important in a world that increasingly relies on artificial intelligence.
• Peregrine Global Services has worked with institutions of higher learning to launch a new Critical Thinking Assessment tool to help schools measure both retained knowledge and acquired competencies.

 
Technology has traditionally disrupted education, and higher education institutions have struggled to keep pace with these changes. However, when institutions of higher education partner with the technology sector, they can become sources of disruption themselves.

One of the most notable examples of how technology disrupted the educational field is the calculator. As Sarah Banks outlines in a 2011 master’s thesis that analyzes historical attitudes about the use of calculators in junior high and high school math classrooms, the invention met with mixed responses from educators.

Some educators viewed calculators as helpful tools that could speed up calculations and save time, allowing students to focus on more complex mathematical concepts. Others expressed concern that calculators would become crutches for students, hindering their ability to develop basic arithmetic skills. Eventually, of course, calculators became indispensable tools in the classroom and beyond.

More recently, artificial intelligence (AI) has emerged as a powerful new technology that has the potential to revolutionize education. However, educators such as Andre Perry and Nicol Turner Lee have expressed concerns about the possible negative impacts of AI. Among other things, they note that its algorithms can perpetuate bias and discrimination. Industry observers such as Lyss Welding point out that AI poses a risk to academic integrity because it allows students to plagiarize and cheat on homework in ways that are easier, faster, and harder to detect.

Artificial intelligence (AI) has emerged as a powerful new technology that has the potential to revolutionize education.

Despite these concerns, AI technology has become an integral part of modern education as more educators are actively adapting and leveraging it to benefit their learners. But teachers should not introduce technology into their classrooms unless they are also helping students develop their skills in higher-order thinking. While technology provides tools to assist with calculations, information access, and other tasks, critical thinking enables students to make sense of that information and use it effectively.

The Importance of Assessment

However, while critical thinking is widely recognized as an essential skill, it can be challenging for higher education institutions to quantify or measure how well students have learned it. Assessment is a vital and dynamic component of teaching knowledge, skills, and competencies. It informs program and institutional improvement, providing invaluable information that administrators, faculty, and staff can use to make data-driven decisions that lead to better student outcomes.

One of the key difficulties in assessing critical thinking is defining what it is and how it should be measured. Critical thinking is a complex skill that involves the ability to analyze and evaluate information, think creatively, and make reasoned judgments, as Richard Paul and Linda Elder outline in their 2019 publication. It is not a single skill that can be easily quantified or measured through traditional assessments. As a result, educators have had to develop more nuanced approaches to evaluating critical thinking skills, such as project-based assessments and open-ended questions that require students to demonstrate their reasoning and problem-solving abilities.

While critical thinking is widely recognized as an essential skill, it can be challenging for higher education institutions to quantify or measure how well students have learned it.

Another challenge in measuring critical thinking is ensuring that assessments are fair and unbiased. Assessments that are overly reliant on multiple-choice questions or rote memorization can unfairly disadvantage students who may excel in other areas of critical thinking.

For these reasons, educators need effective assessment methods that accurately measure critical thinking skills in a variety of contexts. These assessments should use consistent and objective criteria to ensure that all students are given equal opportunities to demonstrate their abilities.

However, building such assessment tools and overcoming the barriers associated with measuring critical thinking places a large and sometimes overwhelming administrative burden on faculty and staff. Unfortunately, there can be a negative impact on student performance when faculty members must allocate more time and resources to handling administrative tasks than to teaching courses and supporting learner success.

A Partnership Between Industry and Academia

The need for critical thinking assessment tools is being addressed through a recent partnership between various higher education institutions and Peregrine Global Services, an education technology company specializing in assessment and instructional solutions. Peregrine recently launched its Critical Thinking Assessment to help colleges and universities evaluate this important skill.

To ensure that the assessment tool would meet the specific needs of the higher education community, the company developed its Peregrine Partner Program, which involved beta testing the tool with programs of varying sizes and types during the fall of 2022 and the spring of 2023. Each educational partner provided valuable feedback on how to present data to help schools make informed decisions, how to remove administrative burdens associated with assessment, and how to foster a culture of quality.

The partnership between Peregrine and the higher education institutions has led to several unforeseen advancements in technology. These include the ability to analyze exam data by course, cohort, or program, as well as the implementation of blind scoring to remove scoring bias. The new tool also adopts an innovative approach to assessing critical thinking and generating the data necessary to analyze exam results. For example, schools will be able to sort and filter data by levels of higher-order thinking.

The Critical Thinking Assessment uses a standardized rubric covering six critical thinking subcriteria and provides institutions with the flexibility to customize the exams to meet their needs. Academic programs can tailor the service to cover specific disciplines and assess varying levels of higher-order thinking. Learners receive scenarios randomly, ensuring a unique testing experience for each student.

The system auto-scores multiple-choice questions, while designated program faculty and assessment administrators use a rubric to manually score open-ended items. The short case studies and scenario questions are written and validated by subject matter experts with practical and teaching experience in each specific discipline.

“The Critical Thinking Assessment helps make assessment a facultywide effort, where everyone has buy-in,” says Melodie Philhours, associate professor of marketing and director of assessment at Arkansas State University’s Neil Griffin College of Business in Jonesboro. “The assessment tool significantly reduces the time and resources required for assessment, allowing faculty to focus on teaching and improving student learning outcomes. One of the most significant benefits has been the removal of the administrative burden related to compiling and entering the data, as the results are readily available after the assessment is fully scored.”

At the Forefront of Disruption

The collaboration between Peregrine and its partner schools will benefit not only the institutions involved, but also the broader field of education. Any time higher education and the technology sector can work together, they will drive innovation and disruption, ultimately leading to better learner outcomes. With the Critical Thinking Assessment tool, Peregrine aims to help higher education institutions assess not just retained knowledge, but also acquired skills and competencies.

In the future, Peregrine plans to incorporate AI into the assessment and build an aggregate pool, so schools can compare their results over periods of time, internally and externally, allowing them to benchmark against schools with similar demographics. Until then, Peregrine is offering the tool to schools as a course-level assessment they can use in their overall assessment portfolio. 

The partnership between Peregrine and universities highlights the potential for industry and academia to come together to address the challenges faced by higher education. It demonstrates that when universities are at the forefront of disrupting education in a positive manner, they can move along with technology rather than lag behind it.