Students today can measure almost everything. We can see our grades, our percentile, practice-test scores, expected examination results, estimations for admissions outcomes, streaks, progress bars, and more and more detailed artificial intelligence feedback. Educational technology today is good at informing students about where they stand.
But What Is the Student Supposed to Do Next?
As a student myself, I have seen plenty of educational tools that could identify what my weaknesses were. But knowing that I scored poorly on a certain topic does little to actually tell me how to fix it.
Say I finish a practice assessment and get a 68 percent.
A slightly more advanced system might be able to tell that I'm in the 42nd percentile, tell me that algebra was the lowest-scoring category, graph my progress, and even predict what I might get on a future test.
That seems pretty cool, except that now the student can close their device and think, okay, what do I actually do now?
The Five-Minute Test
I believe that educational technology should be judged based on one main question:
After five minutes of using this information, what can the student do that they weren't previously able to do?
Does the comprehension score recognize a specific type of weakness?
If an AI reviews an essay, does it know specifically what the student should fix and why?
If a practice test identifies a mathematical weakness, does it identify what the student should do next?
If a college-planning tool gives an estimated chance of getting into a school, does it identify what realistic steps the student could take next?
The difference seems minor, but one focuses on describing the student while the other focuses on guiding them. Metrics are helpful, sure, but descriptive statistics are not the same as prescriptive statistics.
Measurement Is Not the Same as Guidance
Scores are easy to read, easy to understand, and easy to compare – a single number captures a complex process. That is all well and good, but the learning process itself is rarely that straightforward.
Students can receive the same score for wildly different reasons. One student might not truly understand the subject; another might have understood it but was distracted on test day; and a third might understand it but simply work more slowly.
Giving the same advice to students with the exact same result can be as unhelpful as describing a group of dissimilar people in exactly the same way.
Educational technology should focus more on diagnosis and more on actionable items. That idea lines up with research on effective feedback: Hattie and Timperley's model emphasizes not only how a learner is doing, but also where they should go next, while the Education Endowment Foundation recommends specific next steps and opportunities for learners to act on feedback. [1][2]
The Metric Should Be the Catalyst, Not the End-all-be-all.
AI Makes This Challenge More Important, Not Easier
Artificial intelligence can give education technology more ways to personalize support. But at the same time, it gives us the ability to create more information without a strong reason to do so.
Longer feedback is not necessarily better feedback.
More analytics is not necessarily better guidance.
A student might not need a three-paragraph description of a weakness in an essay. Sometimes, the most important thing is to recognize the weakest paragraph, understand why it's weak, and address it.
Good educational AI should be able to prioritize – to recognize the difference between what is interesting and what actually matters. UNESCO's guidance on generative AI in education similarly emphasizes human-centered and pedagogically appropriate uses of AI that genuinely benefit learners. [3]
Design Backwards From the Next Step
If we start with the question, "What information do we show the student?" then we will design systems that tell the user what they want to know.
If we instead start from "What decision should this information help the student make?" then we will get systems that tell the user what they need to know.
If the goal is to help them improve at an activity, the output should lead toward the next thing to practice. If the goal is to prepare for a test, the result should lead directly into the next form of practice. If the goal is to keep them informed, the dashboard should make it clear what they should focus on first.
It doesn't mean scores, analytics, or predictions are useless – they can all be extremely useful and valuable. But it does mean that these systems should be designed to get the student to the next step, whatever it is, faster.
The Metric I Would Rather Optimize
Educational technology often focuses on engagement, completion rates, time spent, scores, and more.
But I think there is another metric that might be more valuable:
After interacting with this technology, is the student more prepared to make progress? Is the student less confused about what they should do next?
That is harder to track than a retention rate, sure, but it might be an equally important metric to track in the long run.
Students already get a ton of information about what they are doing right and wrong.
The next generation of educational technology should not be focused on being better at describing who we are, but instead on helping us move forward.
References
[1] Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. https://doi.org/10.3102/003465430298487
[2] Education Endowment Foundation. Feedback: How can feedback develop independence? https://educationendowmentfoundation.org.uk/16-19/developing-independent-learners/feedback
[3] UNESCO. (2023). Guidance for generative AI in education and research. https://unesdoc.unesco.org/ark:/48223/pf0000386693
[4] Darmel, Alena. Students Busy Using Laptops in the Classroom. Pexels. https://www.pexels.com/photo/students-busy-using-laptops-in-the-classroom-7742816/
[5] Nilov, Mikhail. A Teacher Pointing at a Student's Laptop During Class. Pexels. https://www.pexels.com/photo/a-teacher-pointing-at-a-student-s-laptop-during-class-9159087/
[6] Fischer, Max. A Teacher Talking to Her Student Using a Laptop. Pexels. https://www.pexels.com/photo/a-teacher-talking-to-her-student-using-a-laptop-5212657/
[7] Pexels. Free Stock Photo & Video License. https://www.pexels.com/license/
