ChatGPT Hallucinations Teach Educators Valuable AI Lessons

Trying to train people on technology that isn’t 100% ready for prime time comes with its share of surprises. During our training with River Dell Regional School District in New Jersey, ChatGPT was wildly hallucinating during our sessions.

When demonstrating how to work with documents in ChatGPT, we asked the bot to summarize the Chicago Public Schools AI Guidebook, but ChatGPT instead – quite amusingly – described:

1.  The cost basis for the sale of an estate

2. The performance characteristics of compliance management systems

3. The main character’s connection to Nigerian culture from the novel Half of the Yellow Sun

And indeed, none of these things were in the guidebook we helped CPS develop.

The errors did help the training critically, however – providing not only an obvious way in which this technology still routinely hallucinates, and quite badly at times – but more importantly, as a great reminder that we always need to be critical users and consumers of genAI tools and AI-generated content.

This experience reinforces the importance of:

1. Skepticism: Especially when the output seems off-topic or inconsistent. Cross-check AI-generated information against reliable sources when possible

2. Human oversight: AI is a tool, not a replacement for human judgment and expertise.

3. Flexibility: This technology is still in the early stages, and what may not work today, can work tomorrow. Keep trying (we did eventually get there!)

These unexpected moments in our training sessions ultimately became valuable teaching opportunities. They highlighted the current state of Gen AI technology - impressive yet imperfect - and emphasized our critical role in guiding its use. Thanks to Vito Delaura and the River Dell team for being such good sports.

Want to partner with AI for Education at your school or district? LEARN HOW