Together with Professor and Assistant Chair Dr. Joel Rudin from Rowan University, in this blog article, we explore his approach to using online business simulations as an evidential tool to prove Assurance of Learning as part of his MBA program.
What is Assurance of Learning (AoL)?
“’Assurance of learning’ means different things in different contexts, but above all, it’s a data-based process that demonstrates learning improvement.”
Defined by Joel in the above quotation, Assurance of Learning is a term used across many accrediting agencies and forms a key criterion in how they accredit institutions.
Agencies such as AACSB want to see institutions’ Assurance of Learning plans, what data they’ve obtained, and how they’ve used that data to demonstrate improvement. Joel explained a key element of their accreditation process:
“Once every five years, they send a team of three people, these are Deans of Business from other universities, to review the evidence of learning acquisition in their school’s programs.”
The process emphasizes the collection of data at the core of AoL and typically follows a three-step “loop”:
- Collect Data
- Devise a Plan to Improve
- Go back to Step 1
“They don’t expect perfection and grades scoring 100%. They want to see proof of improvement. When you are proving improvement [in that particular subject], they then want you to focus on another subject to focus on improving that.”
Proving Assurance of Learning is typically done via quantitative data collection throughout a learning exercise. How and what data is collected requires experimentation with various tests and formats.
Where Does the Data Come From?
“We teach in various ways, so why not assess in a way aligned to that teaching style.”
Data to prove learning improvement has evolved throughout the years, starting initially with Nationally Standardized Tests before moving on to multiple-choice tests in capstone courses and then using various tests across the same courses.
“About 20 years ago, there was a preference from agencies such as AACSB for institutions to use ‘Nationally Standardized Tests,’ but there were some quality issues here. They were very expensive and not very effective. We then moved to commonly used ‘multiple-choice tests’ for capstone courses. These are better because their faculty-driven and adapted to the particular courses students have studied previously. They are more tailored, but the problem is they’re harder to achieve improvement. We then moved on to various assessments: we teach in various ways, so why not assess in a way aligned to that teaching style.”
Once Joel’s faculty had agreed to use different tests tailored to individual courses, Joel began looking at ways business simulations could provide this AoL “loop,” demonstrating improvement in students’ learning and knowledge acquisition through quantitative data.
Business Simulations and Assurance of Learning
“The nice thing about business simulations for Assurance of Learning is that simulation providers keep a record of what happens during the exercise: not only who finished first and who finished last but what decisions they made. I can compare and contrast which teams made the optimal decision to improve their company [in the simulation].”
Joel explained two approaches through the use of simulations to prove learner improvement:
- Decisions – what percentage of individuals/teams made “good” decisions for their business?
This way, data is collected and measured to demonstrate improvement based on the level of “good,” informed decisions made. This is based on students’ use of applying strategic models, concepts, and market research in their rationale for what decisions they made during each round of the simulation. - Results – what percentage of individuals/teams made decisions that yielded good results?
This way, data is collected and measured based on the number of points awarded in the simulation exercise. Points are awarded for increasing their stock market price, with their overall points outlined in the simulation’s balanced scorecard. Results-based is currently the approach Joel is using.
“They [Accrediting Agencies] want some successes, but primarily they want to see measurable efforts to prove assurance of learning; they’re not after perfection.”
Used in Joel’s MBA class ‘Operations Analytics,’ the simulation was rolled out in class across ten “teams” of students, taking part in a total of seven simulation rounds. Based on the weekly insight reports provided by Edumundo’s Operations Team, Joel could identify the results of the simulation’s stock market and balanced scorecard after each week.
After the simulation exercise, Joel collected 70 results, of which 57 (81%) showed an increase in stock market value while the remaining 13 (19%) showed a decrease in stock market value. This data was then used to prove Assurance of Learning during accreditation exercises and provides a benchmark for future simulation exercises to monitor improvement increases or decreases.
However, his approach wasn’t solely defined by the need to prove Assurance of Learning.
“I wanted a simulation. I wanted active learning. Today students want a multi-media, interactive experience; they don’t want to be taught the way we were taught.”
A good example of using and measuring the impact of technology is not only for accreditation purposes but for the learning style of a more digitally native student body.
