AI Literacy Curriculum

Mike Kentz's New Assessment Design Framework for the AI Era

The Five "S" Model for Prompt Framework

Generative AI Critical Analysis Activities

Character.ai

Chatbot Cheatsheet

The Skill Code by Matt Beane

Mike Kentz's Website & Process-based Assessment Resource

Mike Kentz's Substack

Amanda Bickerstaff

Amanda is the Founder and CEO of AI for Education. A former high school science teacher and EdTech executive with over 20 years of experience in the education sector, she has a deep understanding of the challenges and opportunities that AI can offer. She is a frequent consultant, speaker, and writer on the topic of AI in education, leading workshops and professional learning across both K12 and Higher Ed. Amanda is committed to helping schools and teachers maximize their potential through the ethical and equitable adoption of AI.

Mike Kentz 

Mike is an award-winning educator with 14 years’ experience across teaching and journalism. He recently gave his first TEDx talk on the subject of AI Literacy and will be presenting at a wide array of conferences this summer. He graduated from Georgetown University, Relay Graduate School of Education, and Wilkes University. He currently teaches at Benedictine Military School in Savannah, Georgia. Originally from the New York/New Jersey area, Mike lives with his wife, dog, and cat in Savannah and enjoys playing basketball and tennis in his free time.

Amanda Bickerstaff
Hi, everyone. Welcome to our webinar, the first of the school year, which is really exciting. We took a little bit of a break for the summer, but if you've been here before, you know, it takes a little bit of time for everyone to get in. So we'll give everyone just a minute. But what we'd love you to do is say, hello, where are you? What do you do? That would be amazing. We have already over 150 people, so I think we're going to be a big group today. I think this is probably what happens when we take a little bit of time off. Everyone wants to come and hang out and talk about AI literacy with us.

Amanda Bickerstaff
We're super excited today to be here with Mike Kentz, who is a former educator in my home state of Georgia, who has gone off, which is amazing. We love entrepreneurs. And so he's now started his own consultancy, Zane Tech, and is doing a lot of work around trying to get us to reimagine assessment in meaningful ways. And I think for us, like, we've been talking about this for over a year now, that assessment needs to change. You may have seen our are slightly viral, like the retirement party for traditional assessments. We believe very strongly that there is going to need to be a shift in the way that we assess students. It's been coming for a long time, and I was really impressed when I first met Mike over a year ago.

Amanda Bickerstaff
I think about a year ago, he was already thinking about this as soon as chat GBT was launched to the world. And so we're really excited to have you all here today. As always, this is a community of practice. We always love our chat because you're going to have as rich of a discussion in the chat as we're having here. So get involved. I love that we already have almost 60 people saying hello from all over the world, some that we recognize, which is amazing, but we've got all kinds of wonderful people from around the world. But also, if you have resources, if you have ideas, put them in the chat.

Amanda Bickerstaff
Like, if you have seen a great example of a redesigned assessment that goes maybe a little bit different or the same that Mike and I are going to talk about today, please share it. But also, if you have a question for us, because we have almost 300 people, I will not be able to follow the chat as usual, as well. So if you have a question specifically for us, what I would suggest is using the Q and a function at the bottom so the chat is for each other. And then we have mandy and Dan from our team who are there to help out. We'll drop some wonderful resources. But, Rhea, if you really have something for us, put it into that Q and A. So what I'm gonna do is I'm gonna come off share, and I just wanna introduce Mike.

Amanda Bickerstaff
So, like I said, mike and I met just about a year ago, and I was deeply impressed by the is thoughtfulness and intentionality around, like, thinking about how to start shifting the rhetoric from generative AI is going to ruin learning, and it's going to only be about cheating. And to, like, how could it actually be used to galvanize and to augment and to excite students around assessments? So, Mike, I would love for you to introduce yourself, and then we'll get started with a little bit of a presentation.

Mike Kentz
Sounds good. Yeah. Hi, everyone. My name is Mike Kentz. I have been a high school and middle school teacher for nine years. Prior to that, I was a financial journalist before switching into teaching, and last year teaching at an independent school down in Savannah, Georgia. I, like Amanda said, was kind of looking at, okay, things need to change. I'm not exactly sure how. How can I experiment to see how, again, I can protect students, but also potentially leverage the technology at the same time? And so what I'm going to present today is going to focus a lot on those experiences. And I'll go ahead.

Amanda Bickerstaff
And just before we start, we will be kicking out some of our AI friends, everyone. Just so you know, we will have the recording for everyone. But if you have an AI transcription, we will be letting those go so that we can really focus together on what Mike is saying. So, do you want to share your screen, Mike?

Mike Kentz
Sure. Here we go. All right, so the name of this presentation, as you know, is stop grading essays, start grading chats. And the subtitle is a process based approach to updating assessments in the age of AI. And the title can be a little bit misleading because the focus on essays sometimes makes people believe that this type of process based assessment is only applicable for an english classroom. But I like this title because I think it's punchy and I think it gets to the point. But I think what you'll see through the course of this conversation is that this type of approach can actually be applied to a wide range of language based assessments. So even science class, history classes, not so sure about math, I haven't quite figured that out. But language based assessments, this can be really applicable.

Mike Kentz
So what I want to start with is an image that I think depicts somewhat, I think what you could describe as the traditional role of the teacher. So typically speaking, we are usually have been at the wheel showing our students how to drive the boat. Metaphorically speaking. We're taking them through the ins and the outs, and they're observing, taking notes, participating, and getting ready for their turn. And then ultimately, in this particular scenario, they're going to step in, take over the wheel. We're going to observe, that's the assessment, and we're going to observe how well they drive the boat. What I found last year is that there's a possibility, not in every assessment, but in a number of assessments, that looks a little bit more like this. In this case, we have the student at the wheel, the bottom.

Mike Kentz
I tried desperately to get this bot to stand over the student with the hand on the shoulder, but I couldn't get it to do it. So if you could picture that would be a little bit more appropriate for what I'm describing. And of course, the bot is acting like, as a little bit of a coach, a brainstorm partner, helping them through the process. But the teacher isn't just stepping back and assuming that the bot is going to handle everything. And beyond that, beyond just monitoring the interaction, we as teachers are also taking notes, evaluating, assessing how the student actually talks to the bot. What types of questions are they asking? How critically are they analyzing and critiquing what the bot produces for them?

Mike Kentz
Because ultimately, you can see in this picture, there's a boat coming at this student, and they've got to be thoughtful about how they're doing this. And so we as teachers, again, we're not just evaluating how they drive the boat, but also how they talk to the bot at the same time as a collaboration or brainstorm partner. In my experience, I did this about four or five times last year in different assessments, in different scenarios. And I found the benefits to be wide ranging. And I've bolded a couple of my favorite on here, because this does address both some of the problems that are associated with AI in the classroom, but also the opportunities. So the big problem that we all know is this issue of AI cheating.

Mike Kentz
And the great part about this approach is that we're not shying away and we're not banning AI, we're actually requiring our students to use it and saying that we care how they interact, we really care about the type of language they use, the type of questions, and the way that they expand on what the AI produces. And this goes a long way towards mitigating the problem of AI cheating by introducing a new part of the assessment that we can control. That we still have some feedback possibilities for another two or three, I'll just highlight on this screen before moving on. In the middle. At the top, you can see what I call the three cs, communication, critical thinking and creativity. And in some of these interactions, what I saw was an opportunity for feedback that I've never really had before.

Mike Kentz
I've tried to grade the process of my students learning before via graphic organizers, via collaborative conversations that they're having with their partners, and never have I had the opportunity to give feedback on these three particular areas in the same way that I did last year when my students were using AI and I was evaluating and assessing. And then of course, right in the middle, I have preserves the role of the teacher. The teacher will always have a role, and there are other roles beyond this too. Don't get me wrong, this is not the only thing, but what this does in the context of assessment is it gives us something a little bit different to assess and it shifts our role and it makes sure that we are a major process of how our students are thinking, problem solving and communicating.

Mike Kentz
So before I get into some of the really nitty gritty details, I do want to start with, if you're thinking about doing something like this, it's actually really important to determine a specific type of task or assessment that's going to produce an interaction that's worthy of analysis. And I think that's a little bit of a departure from where we've been. A lot of folks, when they focus on AI assessments tend to ask the question, how is AI going to help my students? And you can see the subtitle on this screen. It's a different question which tasks actually produce interactions that are worthy of my analysis and feedback. So in the top row you've got a couple of examples.

Mike Kentz
Brainstorming and collaboration are a great task, that your students can do this and you can give feedback, broad based data, and really any task that's non binary in nature, it doesn't have a yes or no right or wrong answer. Instead, you're going to have a lot of ideas that the student needs to choose from. And so what this means is on the bottom row, you can see this doesn't really work that well if a student is doing AI research and they're looking for facts, because again, there's a correct answer there and a wrong answer, and so you're not going to see your students thinking as much in those situations. So closed end binary tasks with a right or wrong answer won't really produce an interaction that's worthy of analysis, and I just wanted to check.

Mike Kentz
Amanda, you have been talking to a ton of educators in this space. Are you seeing similar things? What kind of feedback are you getting from educators?

Amanda Bickerstaff
Yeah, I think the one thing I like about this slide is that it really takes into account the current capabilities of AI. So even when you come down to the AI research, you could flip it on its head, I think, Mike, where you were looking for evaluating the quality of a response more than the actual prompting, because we do know that hallucinations or mistakes are very rampant and everything, even perplexity, which is saying that it's going to have higher accuracy because it's connected to the web. So I think that's really interesting about finding the right topic.

Amanda Bickerstaff
I think assessment, what we've been seeing is that there's a real opportunity in schools around how do we take certain assessments that might be very at risk of students using AI and start to flip them on their heads, but also to take the capabilities, because now you can have an on demand brainstorming partner that is available, does not get bored, does not get annoyed with you. It's just there you can ask any question you want, you can give it as much feedback as you want. So I really love that idea of the brainstorming collaboration, I will say. I think that most people right now are thinking about it from a negative mindset of like, that no AI. Instead of flipping it into. In some cases, AI is not only helpful, but can be a real driver to seeing students critical thinking.

Amanda Bickerstaff
Like, if you go back to the last slide, I mean, the 21st century, durable skills, if you've used these technologies, you know, it's not that easy to do. Like to be a good prompter. It's almost like you have to be a teacher, you need to be a critical evaluator. You need to be resilient, you need to be patient. You even need to be creative. So the idea of that interaction being prioritized and then uses assessment, I think is a wonderful way to start doing this work in a pretty low risk way, which I think maybe you could talk about like that. That idea of like, these are not high risk, you know, summative assessments. That's like a, you know, but this is an idea of like, truly formative that you can do. So I think that's really awesome.

Mike Kentz
Yeah. On your point about formative and summative, I think it can be done both ways. I mean, you can definitely, I've heard of teachers who are just sort of giving verbal feedback on the interactions as they're happening in real time. And that's one way, of course, developing those skills that you mentioned and that I've been saying about metacognition, analyzing your own question asking skills and your own communication and your writing skills, really. And so it can be both. You know, there were some times where I made it a summative assessment, but I think you got to be comfortable with that and your students need to be comfortable with that. So there's a sort of a process to adapting to this where you can definitely lower the stakes a little bit.

Mike Kentz
So after you've potentially chosen an assessment or just a task within a broader assessment, I wanted to put together a roadmap of what you actually do with students. And so this is less about the design and more about the process and the student facing steps. And so the way I have it laid out is that the first step is you've got to teach principles for safe and effective AI use. Now, that is a very subjective conversation and a subjective decision that every teacher should really make based on their needs, their content, their discipline and their task. But I think if you start with a, some basic rules of the road that involve students recognizing, you know, I'm going to ask for, I'm going to ask AI possibly for suggestions rather than asking it to do my work or asking it for answers.

Mike Kentz
These types of things inform how you teach and how you develop your students skills at using AI the quote unquote right way. And I just one last point on number one, I think it's going to be an ongoing conversation for a long time about what safe and effective use looks like. And I think if we can get comfortable with that, sort of comfortable with the ambiguity and the abstraction and the subjectivity associated with working well with AI, this process becomes a lot easier. And I've had a lot of teachers say, how do I grade it? Because it's so different for every. And yes, you're right. If you can start to get comfortable with that, this gets easier. Number two and number five go really well together.

Mike Kentz
I'll do this really briefly, but asking your students to write a purpose statement beforehand that essentially outlines what, you know, what am I trying to achieve? Why am I trying to achieve it? Why do I think that AI might be helpful? And even if your students don't know, asking them to predictions are a really good metacognitive skill. Take a guess. What do you think is going to help here? And then step three is actually having them interact with AI for an assessment grade or a formative feedback mechanism either are fine. The traditional assessment output would come after that. And then the reflection becomes this. Really, frankly, it's a really beautiful experience because what they do is they take the purpose statement and they take their own interaction and they compare them.

Mike Kentz
Did my interaction actually match the purpose and the goal that I was trying to achieve? Why or why not? And they develop a really deep sense of AI literacy or deep skill and AI literacy from that process. What I wanted to do next was present a case study.

Amanda Bickerstaff
Oh, just before. Do you mind if we go back? Sorry to interrupt, but so I think that there are a couple questions around, like, safety and performance standards. So I wanted to, like, maybe take a little bit of a step back from here. Of course, I live in New York City, so I don't know if you can hear the sirens, but it wouldn't be a webinar unless someone buzzed my door or there were sirens. Welcome to being in New York. But I think that this idea of safety is really interesting. I think that you have to start this conversation with ethical use and safe use. Like, number one, do not give up a lot of your information about yourself within these chats. Like, you're not just giving your name and your location and your best friend's name and your parents income.

Amanda Bickerstaff
Like, you know all those things. I think that's one. Number two, being explicit about, like, the ways in which you use AI in meaningful and effective ways. It goes beyond a yes or no question, because we have seen adults that have seen this as not conversational. So actually showing and modeling, I'm sure you do that too, of like, you build the AI literacy and then model best use cases is really important because these technologies don't work like anything that has ever come before. And if a kid, their primary use of technology is like YouTube, TikTok, and Google, that is not how generative AI works. So the idea of modeling it, safe practices and not assuming any level of knowledge, a lot of what we've been talking about lately is that AI native does not mean AI literate. It just doesn't.

Amanda Bickerstaff
The same way that digitally native doesn't mean digitally literate. So the opportunity here is to fold these things in together, model them, and then release your students in a safe space to do this. And so we had a question, though, Mike. We're going to show some student examples in a moment. But can you talk about how you, like, picked the performance standard or the standard you actually worked on with this? Can you actually talk a little bit about when you did this, when it worked or didn't work, how did you actually identify what you were going to be assessing?

Mike Kentz
Sure. Yeah. I wanted one quick point on your safety point because I think it's a really nice analogy. One analogy that I use with my students is stranger danger. I like to say approach AI like it's a very capable stranger, or even a brilliant stranger in the sense, not that it's going to hurt you, but you need to be analytical and critical because we know that when you're talking to someone, even a person you know fairly well, you're not going to accept everything that they say automatically. And so take that understanding and apply it to working with Genai so that you have a certain level of criticality. I just wanted to throw that out there. But to answer your question about how to figure out the standard, the first interaction that I asked my students to do was actually with a character bot.

Mike Kentz
And the idea I really wanted to make sure that they had literacy around character bots so that they were protected, so they interacted with a Holden Caulfield bot after we read catcher in the Rye. So the standard in that case wasn't actually brainstorming or collaboration. What I asked them to do was show active listening, empathy and conversational skills. Because if you're not, if you don't remember that book very well, the main character is quite depressed for the whole book. And so going into it, I said, you know, we're not just going to pepper it with questions, we're not just going to demand things from it. We're actually going to see if we can develop our empathy. And so that became the crux of the rubric was empathy, active listening and conversational skills.

Mike Kentz
Moving forward from that when I had them work with chat GPT, and this actually goes a little bit into the case study. Amanda, do you think I should jump to the next?

Amanda Bickerstaff
Yes, I do have a question from the audience. When we finish this part, we just want to drop the link to the infographic. I don't know if we have that. So just as an FYI, for those of you that want the infographic, we'll make sure that we drop that as soon as this part is done.

Mike Kentz
Yeah, I wrote up a lot of what you're hearing right now into a blog post as well that has all the infographics in it. I, and I think, I know we have that link as well.

Amanda Bickerstaff
Yeah, Bruce, I think Bruce has helped out, so we'll make sure to share with everybody. But yeah, I think the case study is great, but I do. Can we just talk about that. That's really interesting, though, is that you started with, again, a low risk, durable skill focus, and then you started to phase in the more kind of like standard aligned pieces because you're building competency. And also, I think you as a teacher need to get comfort, too, right? Like, you can't. Like, you need to have it be low risk, too, because it's going to be a different way of assessing and interacting. But if you start with something like, very high risk for everybody, it's probably going to be like, oh, no. But if you start with something that's about those, like communication skills or critical thinking.

Amanda Bickerstaff
And I bet also, Mike, if you could talk a little bit about this, I bet you learned more about your students personalities, their approaches than you did and probably most other things. And like maybe a classroom discussion.

Mike Kentz
Yes, 100%. I like to say that I was lucky that I, the first time I did this was in the design that I described where it was a conversation with a character bot because it was much easier to grade, it was much easier to actually look at than if I had asked them to use chat GPT, I would have never had a clue how to even think about it. And it was a really, honestly, it was a very fun experience to see their personality on display.

Mike Kentz
And I was able, I was also able to give them a lot of feedback on their socio emotional journey in ways that I have never in my entire teaching career been able to before because I could see them, you know, in some cases, they were being very empathetic and I could go and reinforce that and be like, hey, I know that's not all. You don't get a lot of opportunities to do that in your day to day life, but you are building a skill that's really valuable in our world. Keep up the good work. And then I also saw students who were kind of doing the opposite. They were like, messing with the bot. They were like bull, like even like, you know, poking the bear a little bit.

Mike Kentz
And I was able to go to them in a very low stakes way and say, hey, I know you're just messing around. I know that this is just a robot, but maybe let's not make a habit out of it. And so it ended up being a really highly engaging process that I was really enjoyed. And then it took me after that was in September or October of last year, and I knew I wanted to do it again, but it took me four or five months to generate the next assessment with this in mind because it took a long time to think about it. And what I ended up landing on was this brainstorming idea. And this was after, just for context, we read the book Romeo and Juliet, and I created on purpose a really broad menu of project based learning choices.

Mike Kentz
And I wanted to be broad and ambiguous so that I knew they would have questions. I wanted them to have a lot of questions at the start about, like, how do I do this? How do I do this? How do I do this? I don't even know what to think of. So that the brainstorming interaction with chat GBT would be as fruitful and productive as possible. And so once I gave them that menu, I taught them principles for effective AI use. This was a mini unit that took about four or five days. And it goes to what you said, amanda, about teaching the safety part first and modeling. That was a really core part of it.

Mike Kentz
I gave them fictional transcripts where I pretended to be a student, and I gave them an exemplar and a non exemplar, and that allowed them to really look at it holistically and make their own conclusions. After that, I had them do a practice round because they had never really worked with chat GPT at all. And I asked them to come up with a bucket list of things they've always wanted to achieve or produce. And I said, you're going to use chat GPT as a brainstorm partner for your bucket list item. And once we had gotten through that, I felt like they were ready to go. And so they used chat GPT as a project assistant. And I'll jump to the next slide where this is an example of what I would consider to be really strong communication.

Mike Kentz
By the way, for context, these are freshmen in high school, and this student did take a prompt engineering strategy that I gave them, which involves giving a role to AI. But if you look at this, and I know it's a lot of text, but if you look at even just skimming it, you can see that my student did a really good job of trying to go from broad to narrow, using that traditional broad to narrow writing structure, and trying to make sure that he was clear with the bot about what he was trying to achieve, and it ended up producing a really interesting conversation. He expanded on the ideas later. And this is sort of an example of sort of an a plus prompt. And just to reiterate this real quick, I didn't just grade one prompt at a time.

Mike Kentz
I graded holistically, much like you would grade an essay. I don't know if you have any thoughts on this, Amanda.

Amanda Bickerstaff
Yeah, we're actually. So we have some really good questions that are coming up. So we're going to drop in the chat our prompt framework, which is we have one for teachers and one for students that absolutely does this. It's set a role. So that idea, like, you give the role to the bot, is actually more important than giving yourself a role. Like saying, like, I'm a student. The fact that it is setting, we call it the five s's. It's setting the context. It's specific in its language. It is setting out the structure, and then finally, it's that sharing feedback component, which is when you talk about the holistic pieces, what we would suggest very strongly is that people will. The more that you.

Amanda Bickerstaff
It's almost like you use a chat to get to your good prompt, and then you open a new chat and you start with that great new prompt. That in a process, too, of it, not just having to be in one context window or one chat window, is really good, because what you'll see is we see this all the time. You ask a question, and it's this idea of generic in, generic out. And so you don't even realize, your students won't necessarily realize, hey, why is it missing? Why is it only giving me one idea, or why is it giving me 50? It's because you didn't specify the number that you wanted. That is such a great way to teach that iterative, resilient questioning.

Amanda Bickerstaff
And so if you can connect the one chat of, like, the thinking chat and then the one chat of the doing chat, we're really going to get to a high quality product. I think even that really helps, too, because you're showing students that process is as important as product. So the product in this case is this prompt and the output. But the process of getting to this prompt is that trial and error, is that resilience, that critical thinking? So I think that's really great.

Mike Kentz
Yeah. Patience has always been a virtue, but in the case of using generative AI, it's even bigger virtue. The more patients you have, I think the better the interactions tend to be for students and for teachers. And I also, whenever I present this, I always try to make sure I don't just show the highlights. I also show some of the lowlights where my students really didn't understand what they were supposed to do. And so one of them, and I'll be honest, when I first saw this, I cracked up my student at the top. He was trying to rewrite Romeo and Juliet in a modern context, and he just asked, what are the main boring parts in the book? And it just, like, there's so many.

Mike Kentz
And the beauty of this is that I actually got to diagnose my students misconception much earlier in the process than I would have if he had actually, on his own, gone through this entire process and tried to rewrite it. It wasn't even the goal of the project that I gave him. So, again, this is the great part about this, and you said this earlier, Amanda, is it can be sort of lower stakes, light touch, formative if necessary, feedback early in the process, and he's asking the wrong questions. And if he didn't have AI, he'd be asking himself that question. Keep that in mind. All this would be happening in his brain, and I would have no clue that this is where the misconception was occurring.

Amanda Bickerstaff
I mean, it's funny, too. I mean, let's talk about, like, is this a 9th grade boy? I feel like I taught 9th grade, and so I feel like this is, like, definitely a 9th grade boy. But no offense to 9th grade boys everywhere, but that idea of it's even further than misconception. It's also showing you very quickly that this is not. That the real world application of Romeo and Juliet is not hitting that student. So I would go even further into not misconception. But this is an example of, like, where we are teaching something that we have to make more of an effort to make it meaningful for our students. We do a lot of just giving kids classics and saying, you need to learn this.

Amanda Bickerstaff
But Romeo and Juliet, if we took the time ourselves to establish how it connects and why it's important, instead of just a here you go, then maybe the student wouldn't think it was boring, you know, and I think that maybe they still would, but they'd be able to communicate why I. They think it's boring. And so I do think that there are these places in which you're taking something that is, like you said, hidden, right? It's an inner monologue that's happening. And maybe they told their best friend at lunch, oh, my God, Romeo and Juliet's so boring. But realistically, you're not hearing that.

Amanda Bickerstaff
But now, because they're having this natural language conversation, you can actually start to dig into not just assessing them, but also learning more and starting to change your approaches to teaching based on this, which I think is going to be necessary, too. It cannot just be putting it all on the students to do a different job of being assessed, but it's also up to us to take these moments in time and actually improve our teaching and our approaches.

Mike Kentz
Yeah, absolutely. I definitely think one thing that could have been better is rolling it out and being clearer on what the actual goal was because he, regardless of whether or not he liked the book, because students are going to dislike certain books. It's just the way it is. But he's also misunderstanding the task. So definitely reflections on my part about, oh, wow, I got to change up some things if I ever do this again. For sure, for sure. And the last student example I'll give is sort of in the middle. This student tried to do the role construction approach. If you remember the last one, the student said, you are a professional historian. And here we have a student who says, you are a 9th grade student.

Mike Kentz
And this was really interesting because this is an example where the student's putting in a lot of effort, trying to do his best to write a really thorough prompt and use the strategy that I gave him. But he really misunderstood the nature of it because the role construction approach, the idea is I'm giving chat GPT, the role of the expert, right? I'm going to say, you are a playwright, you are a script writer, you are a novelist, you are an accomplished writer, whatever it may be. And this student just misunderstood.

Mike Kentz
And he told chat GPT, you're a 9th grade student, which isn't, it's not really going to move the needle, but again, not the biggest deal for me to just go to him and say, hey, let me just explain to you what this would actually look like and does a ton of things, metacognition, communication, and also AI literacy all in one. Yeah, so those are my.

Amanda Bickerstaff
Another really cool thing is you can have them now test out both approaches to see which has a greater quality. So some of this, again, is like assessing the way they chat, but also using it as a teachable moment. We dropped into the chat some critical analyses where we actually have students and even teachers compare two AI generated pieces. But you could have two prompts, one that, where you gave it a role of a 9th grader and one that you gave it a role as an expert and you know, Julius, not Julius Caesar, but Shakespeare and all these things, and see that the quality will shift and change for the 9th grade student. It's going to be written from their voice and it's going to have like a level. It's not going to probably hit the need of their support. Right.

Amanda Bickerstaff
It's going to be something that is probably what they could do right now, which is okay, but we want them to go further and farther. So to be able to actually have them come back and try those out, because we see it so much. The more that you show people how to use these tools effectively, you start to really build the mindsets of this new technology, of that idea of knowing the way that you are specific with your language. Because realistically, that what your students were doing, Mike, is they were being software engineers, because what they were doing is they were coding with words. They were coding with natural language, with English. And so, just like coding has its own roles and syntax and approaches, so does prompting. Like, it actually makes a difference. The order of words you use, it makes a difference.

Amanda Bickerstaff
The numbers and the format, it makes a difference if you give an example or not. And so that is such a really beautiful opportunity, again, of, like, this is the whole thing, which is why we get so excited about this. It is pedagogy, it is assessment, it is learning, but it's also a brand new frontier into how we can create with technology and how everyone can now create with technology as long as they can write or talk. You're in that position now with the right tools.

Mike Kentz
Yeah, absolutely. Yeah. So, yeah, I found this. That's the end of my slides, and it sounds like some questions have been rolling in, and so maybe we can open it up to those. But on the whole, I just wanted to say I found this. I've learned since doing this a couple of times and also sharing it with people and then working with other educators who are trying to do this. It doesn't, you know, it's not perfect. It doesn't work with everything. I think what we're. The stage we're in right now is mapping out the root system or the branches of when it works and how it works. And so we're in the experimental phase. And so, you know, if you're thinking about doing this, think about lowering the stakes a little bit.

Mike Kentz
Like Amanda said, you know, I like to call it dipping your toes into the shallow end. You know, have your students share transcripts, but don't grade all 100 of them. Just take ten and just look at them yourself. Right. You're not giving them a grade the first time. You're just looking at them and considering how you might give feedback. And if you do that, you might start to see some. Some possibilities.

Amanda Bickerstaff
That's great. So let's come off share so people would see us a little bit better. We have some questions. Let's just do the tactical first. Okay. So the first thing is, if your students are using chat Ut 13 to 17 year olds need parental permission. So I'm just going to say that twice. Get parental permission. There are, you know, if you're using Claude, it's 18 and up. Like, just make sure that you're using the tools within the terms of service and you're being transparent with families. So that's number one. Number two is, can you talk a little bit about what you used to create the character bot? And then also, like, were you just using, you were using chat GT, right?

Mike Kentz
No, I actually used character AI for that one.

Amanda Bickerstaff
Oh, oh. So character AI. If you don't know what character AI is, it's the third most popular AI tool after chat, DBT and Google Gemini. And it's 18 million avatars that students can talk with. So you, and it's very popular with 16 and 24 year olds. They really dig it. They'll even call you on the phone, like Abraham Lincoln will call you on the phone if you really wanted to. You're having a day, maybe you want it, maybe you need a little bit of that up thing. But. So use character AI for the character bot and then you use chat TBT for the Romeo and Juliethe, correct?

Mike Kentz
Yeah, yeah.

Amanda Bickerstaff
And then the differences between the two.

Mike Kentz
Oh, yeah, a ton. I mean, Chachi BT is not technically designed to have personality, as I understand it. I'm not a computer scientist or a technologist, but, you know, it will sort of like throw exclamation points in that try to make the conversation more human like. But yeah, you're dealing with a taskmaster if you're going to working with chat GPT directly. And one of the other things I like to say is, if you're working on this, you really bought selection is a huge part of the design process and I think there's, to me, I don't know what you think about this, Amanda, but I see it as three different groups.

Mike Kentz
You've got the character bots and you've got the, I call them vanilla LLMs, which are just chat GPT, Claude, et cetera, but then also custom GPTs, because I saw, for example, a professor who took this idea and he made a contrarian bot and he had his students talk to a contrarian bot and he had them use it for brainstorming and it didn't quite work the same way. And a contrarian bot is actually a much, you know, it's a much narrower, it's almost a personality bot in a way. Contrarian is, you know, so anyway, bot selection is really important and there are vast differences in terms of what they. What their programming is telling them to do behind the scenes. Yeah.

Amanda Bickerstaff
So, okay, I think there's probably four. I'll go further, just so everyone, and then maybe Mandy, if you can drop in our chatbot cheat sheet. But we have the, like you said we would call them, not vanilla LLMs. We call them the foundation models, like the frontier models. And so that's Microsoft Copilot's built off of chat GBT, but it's chat GBT, Claude. And you're talking about Google Gemini are the ones that we would say. Then you have the pies and character, you know, out there. Character AI's, which are designed to have personalities. Personalities, remember, no thinking, no personality, really, but it's been designed and programmed. The third is where you have custom GPTs that you can create. So if you are.

Amanda Bickerstaff
If you have a paid version of chat GPT, you can create a custom GPT, and then any student can use it with that parental permission because they've opened that up to people. So that's pretty cool. And the last thing is, you have organizations like magic School and school AI that have built student facing chatbots that have some support and some prompting, similar to a character AI, to give it a circle figure chat. But then it's also going to be transparent, so you can see all the interactions, and they also flag if a student was being inflammatory or something along those lines. Also, conmigo is a great example. Thank you for sharing that in the chat. So there are lots of opportunities, but because we're all getting started with this, we would always suggest starting with the place that you feel most confident in. Right?

Amanda Bickerstaff
So let's say that you like to prompt on chat GBT, you get parental permission. We would always suggest that, because then it's also what we believe is that, like, you know, chat or the foundation models are your open horizon. Meaning, like, kids can go a million different ways. It's designed to, like, be creative. It's designed to be like, again, it won't get bored or mad at you for asking the same question five times in different ways. In fact, maybe it'll get better. And so those things are really great. So I just want to kind of play that out, though. You don't need, no one ever needs to know 25 different tools. It's better to pick one or two that you feel comfortable with and do that more quickly, because then you're not getting caught in that cognitive load of, what.

Amanda Bickerstaff
What is the right one, you know? And so that's why I think it's just good to kind of yourself feel confident with whatever tool that you use. So there's that. So the next question we had, do you have anything else you want to share with that one?

Mike Kentz
No, not at all. I agree completely.

Amanda Bickerstaff
Okay. The next question I have is from Samuel in Nigeria, which is like, do you think that prompting will negatively affect creativity or, like, students cognitive work?

Mike Kentz
I think there's a risk of that. I do think there's a risk of that because we right now don't naturally or intuitively, when we work with an AI or an LLM, we don't think, oh, I need to be, like, really thorough and thoughtful and slow down and actually analyze my own purpose and things like that. And I see my friends send me screenshots of their own AI interactions and we talk about it. It's a little bit concerning because we tend to ask it for answers and opinions and it doesn't have an opinion. And so there really is a risk if we don't, if we're not very thoughtful about how we write these prompts. Yeah, I do think that could happen, and that's kind of why I did what I did, because I want to teach the students.

Mike Kentz
You should slow down and be really thoughtful about how you do this. Here's some ideas, but ultimately it's going to be about your creativity, your communication skills. So it's, again, it's like with AI, everything seems to be a problem and an opportunity at the same time. And I think prompting falls into that bucket.

Amanda Bickerstaff
And I agree, too. I think you have to shift our mindsets on this, too, that, like, we are, there are new forms of durable skills that will be created, and one of them will be like, deep evaluation of AI outputs where a lot of that cognitive work will happen at the point of, like, being a developmental editor and editor. And so, like, there are already jobs right now that. That we could have and students could have where they're comparing outputs for what is a better output. That's already happening. And so I think that, you know, one of these pieces is that no one's got it nailed. Everybody, we love that we have, you know, 300 awesome people from around the world being willing to be here with us today because you're all aware of this need to learn together, right? And to try out.

Amanda Bickerstaff
I don't think Mike or I are saying this is the way of the future. It's all going to be grading chats, but it's an opportunity right now to start putting on its head that AI is only for cheating, that AI is only bad, but that it actually can be something that is driving critical thinking, creativity, evaluative components, critical analyses that can be really helpful. And maybe that's not how we assessed success over time, but it is how we build AI literacy. It is how we build critical consumption, and it is how we start to understand how students are using these tools and not using these tools.

Amanda Bickerstaff
And so that knowledge that you're going to be both supporting the building up, but the ones you're building, that is really where we see the value, regardless of this, is like the lift and shift, the new way to grade. It's more like this is something that right now can have an Immediate impact, and it also could be the wave of the future, too. But we don't know yet because we're right in the middle of it.

Mike Kentz
Yeah. Yeah. I would reiterate that, and I said it earlier, but I think it bears repeating. I like to think of this as a root system, and it's really going to break off in a million different directions, and it's going to get much more specific, and we're going to get better at this. But it requires experimentation, and that's a ultimately why I'm doing what I'm doing and starting my own consultancy and advisory service is to train people so that you feel a little bit more comfortable experimenting with it. And then if we share the data with each other publicly, we will get better at developing all those school, all those skills you just said and making educators feel a little bit more comfortable.

Amanda Bickerstaff
Absolutely. So we're going to come up on time, everybody. We're going to make sure that we prioritize your time. But there are two kind of quick hit questions. I'll answer one. I'll get the other to you. How did your students do they share their prompts with each other?

Mike Kentz
They shared them with me.

Amanda Bickerstaff
Awesome. Okay, so did you think about sharing them, like, in a, like, we have our prompt library? Did you think about that as an opportunity to, like, collecting the best prompts for students to be able to use.

Mike Kentz
For future reference, you mean? Yeah, if I were still teaching, what I would probably do is pull up last year's and show them kind of the way I did today. And it would be a really very effective way of saying, like, here's some good ones. Here's some not so good ones. So, yeah, the short answer is yes.

Amanda Bickerstaff
And we have, like with our prompt library, the way that we manage that is that the prompt library has an example you can make your own an example you cut and paste. But then we have four or five tips of, like, how to iterate. So if you thought that maybe, like, this would be too much thinking for the students, like, it's really engaging, that, like, iterative component of this. And the last question is that we have this idea of, like, permissioning in your district. We know that there are incredibly strict rules in some places. We do a lot of this work, like, a lot of this work. And so what we would suggest is, though, like, potentially, especially with your high school students, your 11th to 12th grade students, actually talking about, is there a possibility of working with those students specifically?

Amanda Bickerstaff
Because I don't. You know what? This is like, this is a moment where, like, employers are no longer looking for, like, Microsoft Office or Google suite. Like, I know how to use Gmail. If you're old enough, you remember that moment where you put it on your resume, but instead, they're looking for generative AI, they're looking for prompt engineering, chat, GPT, et cetera. So, like, we would suggest, Margaret, like, thinking about how do you, like, start to have those conversations with your leadership, especially around your teach, your students that are going into colleges and or directly to career, that it is something that, like, by doing this in meaningful ways and supported, safe, transparent ways like that, this is something that can be really helpful. And so the prompt library, we'll drop that into the chat.

Amanda Bickerstaff
So, Mandy, if you don't mind dropping the prompt library in the chat. And then I think that you know what we want. First of all, we absolutely love this work. We love to be able to bring together. First of all, can we just say thank you to Mike? I mean, first. I mean, first of all, he is someone that is going out into the world like that as a teacher, and now he's a entrepreneur. So let's just give him love in the chat. That's just like Mike deserved. Not only did he do a beautiful job, he's advocating for, like, really thoughtful, intentional application of AI in terms of both assessment, but really literacy building. So thank you, Mike. You're getting all the love. You deserve it. If you want to connect with Mike, there's a QR code here. We'll drop his.

Amanda Bickerstaff
His LinkedIn in the chat. He also. We shared his, like, all of his great resources and website. You can connect with us as well. But, like, the ultimate goal, though, of what we do here today is exactly this. Like, bringing together a worldwide, legitimately worldwide audience to talk about what's really meaningful. We don't have all the answers, but together we can start building those answers. And so thank you, everyone, for joining us at different. Like, I know some of them, it's very late for you, but also, thank you so much for sharing your insight and your resources with everyone that matters a lot. And this is why we actually start, you know, sharing some of the chat, because you all are community practice. You are people that are getting started or are on the expert lens.

Amanda Bickerstaff
And we just really appreciate you being a part of this journey with us. So thank you, Mike, for being here with us today. We will have a recording shared within a day. Do you have a final thought, Mike, before we go off?

Mike Kentz
No, not at all. I just, I appreciate so many people coming in here and, yeah, I'm just excited to continue experimenting because it's going to be. It's going to be a really interesting couple of years, I think, as we work together to figure this out.

Amanda Bickerstaff
Awesome. Well, thanks, everybody. Good morning, good night. Go to bed, wherever you are. We appreciate you, and we'll see you with our next webinar. Thanks, everybody.