Impact of GenAI on Learning
A recent study from Wharton found that students who use AI tutors with no safeguards may begin to over-rely on the tools and perform worse when access is removed.
This study on generative AI in high school math education found that while AI tutors significantly improved performance when available, students who used a standard AI interface performed worse than the control group when AI access was removed, suggesting AI can be used as a "crutch" and potentially harm long-term learning outcomes.
However, an AI tutor with built-in safeguards (e.g., not just giving students an answer when stuck) largely mitigated these negative effects, highlighting the importance of intentional AI implementation in educational settings, especially with students.
Key highlights of the study:
• The experiment involved nearly 1000 students in 9-11 grades.
• Two AI tutors were tested: GPT Base (standard ChatGPT interface) and GPT Tutor (with learning safeguards).
• AI tutors covered about 15% of the math curriculum.
Results:
1. Access to GPT-4 significantly improved performance (48% improvement for GPT Base and 127% for GPT Tutor)
2. When access was removed, GPT Base users performed 17% worse than those who never used it, suggesting students were tending to copy and paste answers, leading to less engagement with the material.
3. GPT Tutor (with safeguards) largely mitigated negative effects on learning.
This study emphasizes not only the need for a cautious approach to AI adoption in schools, but also the importance of needing better tools with safeguards in place, AI literacy training for all users, and strong guidance for how to pilot tools in a strategic way that mitigates risk.
Its also a great reminder of how important teachers/tutors are in the learning process (no bot is replacing a teacher any time soon).
Check out Stanford’s study that suggests AI tools designed to support rather than replace human tutors can have some positive impact on student outcomes.
Want to partner with AI for Education at your school or district? LEARN HOW
