Slide 8 Prompt

Slide 9 Prompt

Slide 15 Prompt 1

Slide 15 Prompt 2

ChatGPT Conversation

Rubric Prompt

Amanda Bickerstaff

Amanda is the Founder and CEO of AI for Education. A former high school science teacher and EdTech executive with over 20 years of experience in the education sector, she has a deep understanding of the challenges and opportunities that AI can offer. She is a frequent consultant, speaker, and writer on the topic of AI in education, leading workshops and professional learning across both K12 and Higher Ed. Amanda is committed to helping schools and teachers maximize their potential through the ethical and equitable adoption of AI.

Amanda DePriest

Amanda is a Curriculum and Content Developer at AI for Education. She has over 15 years of experience in public education, having served as a classroom teacher, library media specialist, and instructional coach. She has also taught education technology courses in higher education settings as well as professional development workshops for teachers on the transformative power of technology. She is committed to ensuring that students are prepared for the dynamic demands of the future by leveraging the power of technology-driven instruction.

Amanda Bickerstaff

Hello, everyone. We're gonna give everyone just a little bit of time to come on with us because we have a big group. I'm excited to have you here today. This is something that we're so excited about. This is our, like, our kind of, after our first year of, like, doing a lot of this work, this is one of our biggest, newest pieces of content that we're doing, really focusing on. We're gonna be a little bit of a big group, so it's pretty awesome to see. See, we already have over 200 people here with us today. If you've been here before or not been here before, please say hello. And of course, at the exact moment in New York City, I'm going to get a delivery.

Amanda Bickerstaff

So I'm actually going to give everybody just a minute to get in. Please say hello and get started. And actually, Amanda, if you could take people through the setting the scene piece while I let someone into the building, that would be amazing. So we're going to hold on for one moment.

Amanda DePriest

Awesome. Yes. Hello, everyone. My name is Amanda de Priest, not to be confused with Amanda Bickerstaff, but we work together. I've been doing content curriculum development at AI for Ed for the past year now. And in our webinar today, we want to set the scene, encourage you to get involved using the chat as well as the q and a. If you have, like, a burning question you really want us to address, drop it in the q and a. But if you have any great resources or links or further information that you'd like to share in the chat, that would be great as well. We are going to be doing some prompting today, so feel free to experiment with us when we go do some live prompting with chat, GPT and such.

Amanda DePriest

Go to your preferred chatbot and try out some of these prompts we're doing today.

Amanda Bickerstaff

Awesome. And now that we have almost 300 people here ready to hang out with us today, let's set the scene. And so we just talked about how this is your community of practice. Amanda is my community practice. This has been such a great experience. We're both former teachers. We are both coming from a very practical, non technical background. We're coming to this from being educators and education leaders. And so when we've been doing this work, it's been really interesting. And we focus a lot on every part of education, but more real, like, more like short term. We've been thinking really about what comes next, and so we have these tools that are suddenly able to do amazing things, especially for teachers and students. But we have this really big potential impact that we've seen, and we hear a lot of AI is for cheating.

Amanda Bickerstaff

We hear that all the time in staff rooms. Oh, gosh. LinkedIn, buddy, come on, let's get off of LinkedIn. If it's going to be about AI is just for cheating or for nefarious reasons. And what we wanted to do is we wanted to flip that into, like, this is our opportunity and actually, like, an galvanizing action to start redefining assessment. And with things we're going to say today to everybody, you're probably going to be like, of course, like, this is just good practice. Like, this is things we've been talking about for a long time.

Amanda DePriest

But.

Amanda Bickerstaff

But we truly believe that this is the moment in which it's going to be. It's going to force our hand a bit, that we're going to have to do this work more quickly. And so what we're going to do is we're going to just start with, like, it's been a pretty big shift. And I want to say thank you to Amanda, who made this beautiful image for us. We have a graphing calculator for the humanities. So you can see all the humanities there. Love the iconography and this idea that, like, you know, no longer is a graphing calculator just going to help you do math. And, like, you know, we had this huge change where students were hand calculating not just in, like, calculus, but in physics and, you know, chemistry and all these different areas.

Amanda Bickerstaff

And then all of a sudden we started shifting and changing in which students were able to use a tool to actually support the calculation side and then use the tool effectively. And then that was something that was now being assessed and the conceptual understanding was even more important. And that's what we think is happening right now, is that we suddenly can have a calculator chat GPT, or Gemini or Claude can write a pretty darn good essay or book report, lab report. It can go even further than that. It can create business plans. And some of the things that we have asked students to do in these terms of these kind of more authentic assessments.

Amanda Bickerstaff

And so we really want to start thinking about is if you could take a moment and if you feel comfortable sharing, because we already have so many amazing things also. Danielle. Funny. Like, hello, all the AI assistants in the room. But we will be. Apologies, everybody. We will be kicking those out only because this is, you know, we're trying to keep this to the group, but if you can actually take a moment and drop into the chat, whether you're a teacher or a leader, you're a nonprofit professional, wherever you are in the world. What has been the impact of gender AI on the assessment practices in your context? And just that way we can start to really contextualize this. I can tell you from the work that we do is everything from a return to pen and paper in some cases.

Amanda Bickerstaff

And we're actually gonna talk about, like what? Like, like this idea. Well, actually take a second. But the. You're trying to pin a paper. But I'm seeing here is, oh, Victor is doing write speak show, which is really interesting. A little bit like herding kittens. That's very nice. It can be like that. More student feedback. Kevin's talking about, like, we see, like, all these really great things. So the pen and paper there, Elisa using it more for brainstorming, actually making over documents, that's a really, like, strong approach as well, focusing on skills, not just content. And I think this is really where we start to see some interesting places that we're going to really cover today. And again, I love our community practice that's built in our chat, so keep it coming. I'm actually going to turn over to Amanda.

Amanda Bickerstaff

I mean, Amanda was, first of all, teacher librarian. We love our media specialists and then also an instructional coach around technology. You've had to do this before, so everyone here has had some correlates as well. Do you want to talk a little bit through that?

Amanda DePriest

Yeah. So it's helpful to think about where we've been before when we've had these kind of seismic shifts in education. Like in the seventies when calculators were introduced, there were math teachers protesting in the streets with signs that this was going to destroy students math understanding. And actually, we found that it actually enhances because it frees up, once they have that good grasp of foundational skills, it frees up cognitive space for more advanced functioning. Then when the Internet came around in like the nineties, early two thousands, like, everyone was afraid that no one was going to learn anymore because they could just look everything up. But we know now that the Internet has been a tool to increase collaboration, access to knowledge. It's democratized learning. We've got these online courses now.

Amanda DePriest

Kids can go to college from places that they never could before because of geography. So it's been a huge positive force for learning. And then when we got Wi Fi, which enabled one to one devices, a lot of schools went to that. My district went to Chromebooks. Some places have iPads and things like that. That has allowed a huge shift to, like, 24/7 access, being able to submit work digitally. We wouldn't have been able to navigate Covid without something like that, you know, for better or worse, how we did with COVID But so we've got kind of a precedent for how to navigate these massive shifts. And so it's helpful to kind of draw on that and know that we've been through this before. It hasn't destroyed education. We've adapted, and we've found ways to make it work.

Amanda Bickerstaff

That's really important. And Chris is saying he still has a slide role. I still remember the compass and all the things. Yeah. I think that the underlying point of this, though, is that it is where we've been before. But what's happened, though, is the reason why we're all here together. There's 400 of us hanging out at the end of a school year, is that the time scale has collapsed. It took decades for calculators to make it in from those. Like, even to get to the point where people were saying, you know, protesting on the street. It took years for the Internet to make a difference in our schools and also devices, even though those were in students hands. It took like a decade to really start to be something that we really saw be a huge impact in our schools.

Amanda Bickerstaff

And so I think that what's really interesting about today, though, is that because tragedy t and these tools has such a consumer focus, which means that we all can access those all the time, and so can our students. In fact, not just chats Vt but the most common tool for young people with generative AI is actually Snapchat AI.

Amanda DePriest

That they use not just for schoolwork.

Amanda Bickerstaff

Sometimes they use for relationship advice or what should I wear to this webinar? And so these are the things that are really important for this conversation, is that we have to have these conversations so much faster than we've ever had before, and we have to also recognize a couple things. And, I mean, this is what I just think is so important verse to talk about AI and cheating. Let's just, let's just call it is we've had cheating in schools since Socrates time. It's probably like a stylist and a piece of marble. Like, you know, it's something that's happened forever. And it's happens for all kinds of reasons, like kids cheat. I mean, Amanda, why do kids cheat? Like, what are some reasons kids cheat? Is it because they're all lazy?

Amanda DePriest

Well, I mean, maybe some of them. But I think a lot of them. If you're a perfectionist and you are afraid of failure, if you have high levels of anxiety, if you want to keep up with your peers, maybe you just didn't have the time to study or prepare the way you wanted to. I mean, I think it's not entirely fair to ascribe nefarious motives to kids. I think we consider the whole child in the whole context. It's a very broad issue.

Amanda Bickerstaff

It is. And I think that, again, like, what about students that feel marginalized where they don't feel like they have a space in the school system itself? Like, why does this matter? Like, what if I have no teachers or the society doesn't care how I do? There are all kinds of reasons cheating. And sometimes like kids are like the cheat are like gonna like create stuff. They're the like the most amazing, like effort put into cheating, sometimes more than doing the work. That's always my favorite. We're gonna like spend like a really long time figuring out how to cheat. And then if you just done the assignment, you've been done in half that time for good. Use your powers for good, everybody.

Amanda Bickerstaff

But I think that what's really interesting is that we see that in high schools there's incidences of cheating behavior about 60% to 70% of high school students. And this was done by Stanford. Victor Lee and his team, who are great, have done this report. And what's really interesting, so I was on a panel with Victor this year in Stanford. He was talking about how he looked at other areas of the world in terms of scams and phishing. It actually went up like 1200 percent after the first year of chat CBT being released. So there's this idea that so would cheating was going to go through the roof. So they went back to the same schools to do this research they've been doing for a decade, adding questions in and around generative AI. And what they found is there was no increase in cheating.

Amanda Bickerstaff

Instead, kids were cheating differently. And we see that through search volume for things like, you know, essay Mills or homework help. But what we see is that students are now using trash BT to do this, but they're also using it in many other ways. So we wanna just establish that, like, yes, it is happening, but like, that is not just an, it's not a tool issue. It is a school issue that goes much broader and much larger to the point of Amanda, that idea, that whole child. And so we just really hate that rhetoric. Like, if that's what's happening. Like, we have to get past it.

Amanda Bickerstaff

I know this group is the group that's gonna help us do that in your own schools and systems, but we just want to kind of put a point on this and just talk about what's not the answer. And so, like, let's just call it like it is. Like, AI detectors are not reliable. In fact, you can take the same piece of writing, add a couple spelling mistakes, put it through different AI detectors, and it's going to be AI in one AI? No, not in the other. They're super, super unreliable, and they also can be biased against non native english speakers, but also against students that are trying really hard. So this idea of purple prose that a kid is going to make the perfect essay before college.

Amanda Bickerstaff

So there was actually trying to use this at Stanford, AI detectors around student essays, and it was super ineffective because the kids were trying so hard. They all look like AI because they had the same level of complexity and the same level of try hardness that sometimes chat dvd can have. And they had to show that it wasn't going to work. So we definitely do not suggest AI detectors as an area that you should be thinking about as part of your toolkit. Amanda, do you want to talk about why? Like, so if architectures aren't working, like, why pen and paper is also not the answer?

Amanda DePriest

I've seen so many comments. If I've seen this comment online once, I've seen it 100 times. Well, I'm just going back to pen and paper. I'm going to blue books. We're just doing everything in class handwritten. And I think that does a disservice to a lot of students. Obviously, the students with learning disabilities, things like dysgraphia and things like that, they puts them at a disadvantage because they've come to rely on those accommodations through technology, speech to text, sex to speech, things like that. But then also, a lot of just like, neurotypical students have issues with writing. Today. My son cannot write coherently to save his life. And so that shifts their focus from the content and demonstrating their understanding to writing. And a lot of them will truncate their response. They will write something short and simple to be done with it.

Amanda DePriest

And so it's not going to give you the full picture of their understanding for the most part.

Amanda Bickerstaff

Yeah. And the thing that I'm really fascinated by is that, like, let's say that you go to pen and paper and you have like, kids that love writing and kids that are okay with writing, kids that hate writing for different reasons. Whether it's just like a. Not a flow state or they have this dysgraphia, dyslexia, like, they're really struggling. What happens to all three of those groups? The kids that struggle with, like, a learning disability or lack of flow state will be super discouraged. Right. It's going to feel really hard. The cloud of load will be through the roof and dissatisfaction and also potentially issues with self esteem. And can I do this, number one? Number two is you got the kids that kind of are okay.

Amanda Bickerstaff

That is, that kind of okay could lead now to that other area where we start pushing them into writing isn't for me. But the third group, the kids that love writing, putting them in a pressure pot and saying, you have to do it in this way, will also have negative effects. And so the idea of pen and paper being appropriate in certain areas, and we are going to suggest that there areas where you're establishing students voice or conceptual understanding that it does make sense to have it be non technically supported or at the end where they're thinking about novel applications, or we're going to do some summative assessment. We believe that is really a positive, strong way to do this.

Amanda Bickerstaff

But what we're saying is that to go fully to pen and paper, not only is it untenable for the students, but it's unattainable for you, too. There's no way that you're going to be able to grade that many things. Like, let's be real, everybody, even with AI graders that are still pretty unreliable, you're going to have some problems doing that. So let's just not. Let's just. That's not the answer. And so what we do want to talk about, though, and this is my favorite thing that we found, like, in terms of a construct, to start thinking about this through. And, I mean, I'm gonna let you talk about this, but I really think this is really interesting about this idea of, like, desirable versus undesirable difficulties. So, like, we talk about redefining mastery or cheating.

Amanda Bickerstaff

Like, we really think what it is you have to lean into the hard work of learning, not just the hard work, and that those things are different, that there's hard work that for students that do not help them learn, and then there's a hard work of learning that actually do change the students learning, their approaches, their metacognition, like, and that has to be separated in a meaningful way for the. We think actually, for the first time ever, you kind of have to do it. So, Amanda, you want to talk about this, really do.

Amanda DePriest

Yeah. We came across the research of Robert Bjork out of UCLA's department of psychology, and way back in the olden days of the nineties, which I think it's still relevant today, he defined this term desirable difficulties, meaning obstacles or challenges that are beneficial for the learner to really grapple with because it stretches them, it causes them to grow and enhances long term retention and understanding. And his work goes very in depth of, like, how to build in specific desirable difficulties, things like spacing and interleaving. But what struck me, looking at it through the lens of AI, is this is the thing that a lot of teachers get hung up on, because AI offers an opportunity for students to offload those desirable difficulties. Something is hard.

Amanda DePriest

I go put it through chat GPT, even if I could have grappled with it and figured it out myself, it's just faster and easier to go to AI. And we understand as teachers just instinctively that is going to affect their comprehension. And so where the flip side of that, Amanda, if you'll. I think unless you have something to add, I think we need to look at the other side of this coin, which is that, yes, AI can help them offload desirable difficulties, but it also offers an opportunity to work around undesirable difficulties. Not all difficulties are created equal. Right.

Amanda DePriest

If there are obstacles to learning that are not connected to the child's control, things like learning disabilities, language barriers, executive functioning issues, maybe just plain old immaturity, things that are disconnected from the learning objective, that are getting in their way, like they're having to do written expression, or there are time constraints that are making the child unable to perform. AI can really swoop in and be the hero and help address those. And so we want to encourage people to be open to identifying those instances of undesirable difficulties and be willing to maybe allow some AI usage to counteract those.

Amanda Bickerstaff

Absolutely. And what I love about this is the idea of what this, the whole goal of this webinar is about redefining assessment, and we will be doing some live, hands on, like looking at how to actually adapt assessment. The reason why we're starting here is that it's our opportunity to think about the ways in which we can use AI to augment and support the students. And so I've actually been talking a lot about this idea that artificial intelligence has been assistive technology for decades, supporting students with ieps and 504 plans are non verbal or low sided.

Amanda Bickerstaff

Our students that are struggling those manners, that this is the opportunity for generative AI to be supportive, be assistive technology for everyone, because there are these places in which these undesirable difficulties can be like, we can actually have these support us in the ways that are meaningful, whether that is, we have something around language barriers or learning disabilities. But there are also some really interesting things around just time limitations. This one's something that I've been thinking about a lot. So it's really funny. Chat is really bad at it doesn't do timing well. If you've ever done lesson plan, it'll be like, this could be done in 20 minutes, and you're like, that's going to be three days. We all have teachers also.

Amanda Bickerstaff

We sometimes really struggle with knowing how long with this group of students, this activity, this lesson is going, this unit's going to be, what we often do is we end up running out of time, but then we still expect our students to do that same final project in less time. And so what we've done is we've added an enormous time constraint and expecting the same outcomes. And this is an opportunity where we can actually use generative AI to help us understand that and maybe pare down the assessment. If you want it to be AI resistant, meaning no AI use, or students are able to use the AI to help them in specific, intentional and disclosed ways to actually have them be able to do that high quality assessment, then the time is afforded to them.

Amanda Bickerstaff

And so those are really good examples of, like, how you could either look at it as, like, using generative AI to, like, improve, like, the opportunity for students through, like, your actual lesson planning, assessment planning, or giving them the opportunity to actually have extra support. Because what we've done is we've collapsed the timeframe. What often ends up happening is we kind of end up really just checking for that knowledge. But also, how well do they do time management and planning organization, which are important, but often at a summit assessment? That's really not the goal of that assessment. And so these are the opportunities we can do this. Another place I just want to kind of underline, is that writer's block? Oh, man, who procrastinates? And I can't see you, but I'm going to say me if you procrastinate. Like, we procrastinate.

Amanda Bickerstaff

Like, not only that, a lot of times it's actually not procrastination. It's just like a writer's block or this, like, inability to get to the place of, like, this place and space of cognitive load. And so I was in Iowa this past week. Shout out Iowa teachers and leaders. And someone was talking about how their student actually used SHBT to help them write the first paragraph. And then what they would do is they would take the first paragraph and then that would help them work through the whole document, and they would actually not even use the first paragraph, but they would take it as, like, this inspiration of, oh, that's how I could get started. And then they're using that to jump off from almost similar to an exemplar. I don't know. Mandy, do you have any other, like, thoughts around this piece?

Amanda DePriest

Yeah, I'd like to add, like, I suffer from writer's block a lot and procrastination, mostly because I'm a perfectionist and I want it to be perfect right from the gate. And so I get kind of paralyzed at the start with, like, what's the perfect way to do this? And I'm sure there are many students that are the same way, but oftentimes I'll go to chat and just, like, get me started over the hump, and then I can take it from there. Like you mentioned, a lot of times I don't even use what chat like, presents, but I'm like, okay, yeah, that gives me some ideas. So I think it's a worth considering allowing students who are developmentally ready to use chat GPT in that capacity. Capacity to give it a shot.

Amanda Bickerstaff

So here's what we want to start thinking about when we start shifting into this idea of, like, actually, like, what we should be building in terms of this idea of how to support assessment. Like, and I think this is the thing that we really want people to recognize or to least start thinking through is idea, like, actually starting to identify which concepts or skills that students need to be grappling with in order to meet this learning goal. This is standards based teaching. Everybody, come. Like, we're not telling you anything you do not know. But this idea, though, is that we have to kind of be more intentional with what we do. Sometimes it kind of gets muddled or we have something that we've been doing for ten years and we haven't really.

Amanda Bickerstaff

We've lost the thread in terms of what really needs to be done, and it starts to be a little bit more muddled for us, but also for our students. We need to make sure that our students really understand the why. And I think we're going to see this in ways that we've never seen before, which is like, why are you asking me to do this? If chats, VT, or other tools can help me do this, and I'm going to need that for career, why am I doing x? And in the olden days, or parents maybe you said because I told you so, or it's important, or you need it for, you know, like we always say, like why do you need this? And we always have potentially a pat answer.

Amanda Bickerstaff

We are really going to have to start to say like, yes, you were going to be able to potentially use these tools to augment what you're doing, but without the foundational conceptual understanding, you can't, you are not going to be able to accurately edit, audit, understand, analyze, choose based on how you use the outputs that you're using. Chi two t for the third piece is open to actually considering Genii use if it is not essential to learning goal or that it removes undesirable difficulties that manna just said. And I really love that there's so many people in the audience that have also thought about this as well, because what they're doing is they're starting to think about where the spaces in which this can be used.

Amanda Bickerstaff

And also when you start to make this more transparent and more open and creating spaces when students can use it, they're going to be most likely more open to not using it too, because it's going to start to make sense to them if you do it in a meaningful way that these things, this is why I'm doing this and this is why I need to do it on my own, one to one. And this is when I need other tools or other resources, whether it's people chat or other tools. But the idea that you start to create those spaces for both is really important. And then that last piece is that transparency. We really, really want students to feel comfortable asking if they can use the tools and to be able to have an open conversation without it being immediate, like, no.

Amanda Bickerstaff

That reaction of, no, you can't being like, but I'm running out of time. I need some help with brainstorming. No one's available. Do this with me at home. Is it okay if I use it like this and then I will show you what I do? Starting to have those conversations about transparency is really important. We believe that also if teachers are showing like, hey, we adapted these assessments with generative AI and we're being transparent about use, that it creates a space in which students feel more comfortable asking and using these tools more responsibly instead of feeling the need to hide them or use them in those unethical ways. And so the idea of everyone's giving us a little bit of the idea of olden days. Not olden days, too long. Right. It's like the last couple decades.

Amanda Bickerstaff

But we are kind of moving pretty quickly. So let's talk about this before we actually get into some hands on prompt engineering. So, Amanda, you want to talk about this, like, idea of shifting the definition of assessment?

Amanda DePriest

Yeah. So were thinking, and, you know, we could talk for hours and hours about the definition of assessment and what it was and what it should be, but we think kind of some key pieces of it is that we want to shift it from just the process of measuring students ability to recall and apply specific skills at a certain day and time under a certain set of circumstances. Like we have the test on Monday, you sit down whether or not you had a good breakfast, a good night's sleep, maybe you're in a bad mood that day. That's going to be the measurement of your understanding. Point in time assessment.

Amanda DePriest

We want to shift away from that to kind of a new definition of assessment as an ongoing process of evaluating their conceptual understanding as demonstrated through durable skills or 21st century skills or future ready skills, however you want to call it. And so this shifts it from, like, a point in time snapshot of what you can recall at that moment to, like, an evolving understanding of how you are viewing these concepts, like, in a whole context, if that makes sense.

Amanda Bickerstaff

Absolutely.

Amanda DePriest

And then I think, yeah, we get into the durable skills on the next slide, Amanda.

Amanda Bickerstaff

Yes. And so these are durable skills, critical skills, essential skills, 21st century skills. We love our names of things, but what we're really looking here is what we think is going to really like kind of last the test of time. That even with augmentation with generative AI, these things are going to be very important. Critical thinking, metacognition. I thinking about thinking is actually going to get significantly more important as we go forward, but also things like growth, mindset, fortitude, or resilience, those are really important. I also think about collaboration as collaboration between people, but also collaboration with technology and the idea that you're kind of using it in ways in which when we collaborate, we don't take over. We don't just lift and shift like, we don't say, hey, Amanda, do. What we do is we go back and forth.

Amanda Bickerstaff

Collaboration with students, being able to use these technologies in meaningful and responsible ways, I think, also relies on some collaboration. And so I think that this is where we really get to the interesting places, and we know that it's really interesting. I'll actually say that I'll throw a little bit of a wrench into this idea that durable skills of what people have been looking for and in Korea and industry for the last decade, where we know that these are happening, where this mercury succeeds piece around, these top durable skills really matter. Even those that are at risk of automation are less likely to require these durable skills. We're starting to see, though, that not only are durable skills important, but AI related skills are also becoming more and more important.

Amanda Bickerstaff

For example, John Deere is saying, we want kids to come out and have critical thinking, you know, growth, mindset and ability to learn and passion for learning, but also they need to be able to start using AI to support their own work. And we're going to use that in responsible places. And we actually see that there's been an increase on LinkedIn of both posts, LinkedIn job posts, and people's profiles talking about the need for AI skills as well as having themselves. And so I think that this is where we're going to start to see some really interesting places in which, yes, durable skills, but also AI literacy and the ability to use AI in meaningful and augmented ways can be really important.

Amanda DePriest

Yeah, and I'll just quickly shout out our partner, Vera Kubero, that we are fortunate enough to work with, recently shared on an episode of I think it was the AI and education Conversation podcast, anecdote in which a teacher who is interviewing for a curriculum position was asked to share her screen and pull up chat GPT and given a performance task to complete live as part of the interview process. And fortunately, she had the skills to do that and she secured the job. But imagine being a student who had not been like, taught how to use AI tools responsibly and effectively being put on the spot like that. You would be panicking.

Amanda Bickerstaff

Yeah, absolutely. And I think that's where it gets really interesting. Like, we are going to start to see these. Like, I think it's going to take a while because industry is slow as well in this adoption because it's happened so quickly. But we are going to see more and more where these skill sets are going to get a hiring exercise, going to be like this amount of time at this amount of time, and you're going to be expected to be able to use these tools effectively. Last thing we're going to do before we get some live prompt engineering is we're having a retirement party. You might have seen this on LinkedIn. We are like, it is a joke, everybody.

Amanda Bickerstaff

So hopefully you take it as a joke, but we are this is an opportunity to kind of retire some of those outdated, traditional assessments that we use all the time, sometimes as crutches. And so, like, the idea that, like, you know, that these tools and assessments that focus on lower order thinking skills instead of durable skills or disconnected from the future of work are often not just like students are. Sometimes they don't really assess the thing we want them to assess, but they're also super vulnerable to AI misuse, because, honestly, there are a lot of five paragraph essays that are on the web that were used to train these tools. And so it's something that generated AI chatbots could do. And so this is a little bit of a joke, but we're gonna have a nice party. We're gonna have a cake.

Amanda Bickerstaff

We're gonna go into a golf course and have a nice time. But we want to start to see how we can disrupt these and actually start to augment them. So here are some examples of what we can do, and we're going to really shift focus into this next 15 minutes is really going to be hands on. Prompt engineering we're going to do is, if you want to open up chat or Gemini or cloud, your favorite chatbot, what we want to do is we want to have you work along, and we'll drop in the prompts that we're using. But what we want to do is actually look at starting to think about process over, product, the ideas of actually splitting things up into multiple rounds of feedback requiring self assessment and reflection. I'm telling you right now, generative AI is not good at opinions.

Amanda Bickerstaff

And, in fact, it's really funny of anecdote of a student, a teacher that asked students to write, like, essentially, like, how did this go? And someone used chat, and it ended up being like, this three paragraph, amazing thing. And he's like, I just wanted you to write a sentence. Just tell me. Good, bad, indifferent. I felt comfortable, and it was so funny because I think that opinions actually are very hard for gender Vi to do well at this stage. Doesn't feel authentic, isn't authentic. So that's an opportunity as well. And then that idea of, like, novel application, like, let's do it in ways that have, you know, are really fun and interesting. Whether that's a debate, very hard to use generative AI in a debate.

Amanda Bickerstaff

You know, the idea of modeling things, they could even do teaching where they teach it back, but there are lots of really good opportunities to do this. So what we're going to do now is Amanda is actually going to share her screen. And she's going to. We're going to go through kind of like a think aloud, which we love to do here, of where we're going to take you through, kind of taking everything we've talked about in terms of desirable and undesirable difficulties, this idea of durable skills, and we're gonna actually look at an existing. It's an essay, right? About the great gapsie. I'll tell you, everybody, that was what I chose to do in my AP English exam, and I could not remember anybody's names, so I did.

Amanda DePriest

I got.

Amanda Bickerstaff

I passed, but not well. So I definitely remember that. Not going so well for me. But what we're gonna do is Amanda's gonna kind of show you through. Can you make it slightly bigger, Amanda? So if you can just make the screen image slightly bigger.

Amanda DePriest

Yeah, let me.

Amanda Bickerstaff

Whoops.

Amanda DePriest

Hang on. I lost all my keyboard shortcuts when I switched this, like, remote keyboard. Things are different now. Okay, so can everyone see my chat GPT 40 we're using today?

Amanda Bickerstaff

We're good.

Amanda DePriest

All right. And so this, we want to kind of take you through a whole process of, like, how you might start from. Well, I have this assignment that they always do on the great Gatsby. So I'm going to copy a slide or prompt. There it is that I had ready and put it in there. And I'll read off the start. It says, act as an experienced high school teacher skilled at modifying instruction to accommodate student needs. Your task is to review the assignment instructions and triple quotes below and identify any issues with the assignment that are unrelated to literary analysis and might prevent students from demonstrating their learning. You could also attach this as a file if you have that functionality in the version of the chatbot that you're doing.

Amanda DePriest

I offset it with triple quotes here so I could get a quick copy paste. And then I give it some examples of things that might prevent student learning, including a language barrier, learning disability, or executive function issues, things like that. And then all this below it is the assignment instructions that I would normally give to students. And it's basically just write a paper analyzing the major themes of the great Gatsby. These are their instructions. Select a theme, write a thesis statement, give text evidence. Like, it's. It's a whole paper that they have to write. They have to cite using MLA. It needs to be four to six pages. It's a traditional assignment, 100%. And so chat GPT is.

Amanda Bickerstaff

Can we just like, we just put that into the chat for everybody? Can you just like. Hopefully it'll be big enough. If not, we'll create a document. Well, I'll share the document.

Amanda DePriest

This is going to be a long one. I think it might be too long. It's not letting me paste into the.

Amanda Bickerstaff

What I'll do is I'll put this into. Yes. And so, yeah, Chandler. I'll just make it right now, everybody. I'll just share the document. So just give us.

Amanda DePriest

We can share the link to the document. It's set up so that anyone can look at it.

Amanda Bickerstaff

So we are good at that there.

Amanda DePriest

Okay. So it identifies right away some of those areas of undesirable difficulties. So there could be a language barrier. We have students who are english language learners. They might struggle with that, so you might want to accommodate for that. Learning disabilities, executive function issues. These are what I suggested to it, but then it gets into some other things. Access to technology. Because the essay is required to be submitted through canvas, it assumes that all students have reliable access to technology. And the Internet, which we know is not the case. I know I live in Arkansas, where a rural area, and access to Internet is a real issue for us. So this could be a problem. Time management.

Amanda DePriest

We know that the prefrontal cortex, the part of the brain that manages, like, you know, decision making and planning and things like that, does not fully develop until you're around 25. So these sweet high schoolers, bless it, are working with a squishy prefrontal cortex, and they may have issues with time management and executive function that have no fault of their own. So this assignment, as written, includes a draft submission for peer review one week before the final due date. That's a lot for them to have to keep in mind. So if they have poor time management skills, they might struggle with that. Peer review is going to be hard, too. In a previous version of this, it identified that because some peers might better at giving feedback than others, some people might feel insecure about it.

Amanda DePriest

I think that's addressed in the emotional and psychological factors here. So it's identified several things that I didn't think of when I designed this essay prompt. And actually, I'll be forthcoming. I did not design that assignment instruction chat. GPT wrote that. But so now what I can do is, I can say, redesign the assignment instructions to accommodate the issues that you identified.

Amanda Bickerstaff

That's really cool.

Amanda DePriest

And now I'm going to get a whole new deal, which, Amanda, I think you were actually going to explore in more depth, so I won't break this one down.

Amanda Bickerstaff

Yeah, that are coming in the chat. So just as you know, our prompt library is designed to be something called a one shot prompt, which is why the prompt is quite long. You could do it as a chain of thought where you actually break it down into sections. We don't have a ton of time together, unfortunately, everybody. But we could have done this where we have the assessment that we upload and then we take it chunk by chunk. We do this also to start to show you what's possible. But the more that you do with prompting, the better off you're going to get at starting to think through what works best for you. There are also some issues around, like that these tools do have an environmental impact.

Amanda Bickerstaff

Of course they do, but they also, I mean, at this stage, we are hopefully going to be working. The companies themselves are working towards hopefully more sustainable practices. But I do think that this is the opportunity to start to, like, think through and show you all as a demo in terms of the way that we do this. But yes, we've done it with a few shot and train of thought, but our prompt library is designed to be one shot so that people really can start to see what's possible. But let's go back to the top here and agree, Laura. The energy intensive is going to be those imagery pieces. So here we go. So we asked to actually start to think through, actually redesigning the assessment instructions to accommodate the issues.

Amanda Bickerstaff

So what we could do also is we could pick one of them that we want to look at, but the idea of actually starting to think of going through this is like actually going through and select having this be nailed down. And what we can do, though, is, let's say you're an early teacher and you're not sure or you're not quite sure and you want to double check. Amanda, can you ask chat bot why it made that change? Like, how this. How this change actually is going to meet the undesirable difficulty. And, Chris, we'll definitely share the chat with everybody afterwards. You can kind of see it afterwards. So here we go. So this is. They have alternative formats for the final product. The length and format flexibility has changed. There's support for reading and understanding some organizational tools and some feedback options.

Amanda Bickerstaff

So I think that this is where you can start to also use the processing. Like, you know, this happens really quickly, and it can be a little bit hard to kind of nail down exactly what the changes are or how. But I think you can start to start looking, especially if you're using the frontier models of not just how do I adapt it, but, like, what's actually being adapted so I can start to lean in and create differentiated materials because I might. Now instead of having these one big summative assessment that everyone gets, I have versioning of that for those that have less time, those that have the need for more conceptual support, those that are going to have special, that have disability, learning disabilities. But that's where you can start to get really engaging and interesting about this.

Amanda Bickerstaff

And so, I mean, I wonder if what we could do is start going through and like focusing on, okay, let's build a similar version that's just for actually creating something more process focused over product, force focused, which is that checkpoint piece.

Amanda DePriest

I'm going to say break the assignment. Whoops. Can't type down into an incremental process with clear steps for submission along the way. It's breaking it up by week. So they have a, they have to choose their theme by the first week. They have to develop their thesis statement and gather text evidence in the second week, they're drafting in the third week. Maybe I don't want to spend this much time on it, but like I could rewrite this as, okay, it's not going to be weeks. It's going to be, you know, days or units of two or three days or something like that. I don't have to take it exactly as given, but if I do want to spend, I think it's five weeks on it. Yep, I could.

Amanda Bickerstaff

That's pretty long. Why don't we wait? This would be where I think specificity, like, as you know, like this is definitely going to be specificity of language are going to start to really like be important. And I think it's really interesting though, because we're getting some feedback from the audience about this idea of UDL. And absolutely there are great, I mean, UDL is a evidence based, amazing way to approach both assessment and backwards planning and learning design. And you can do this with, it doesn't have to be a crutch. It could be something that actually you can use to support your own UDL pieces or planning. But we know that a lot of teachers are not doing that right now and also can use this to actually start to create spaces and what just UDL may never have.

Amanda Bickerstaff

They never had support to train to do it or really good exemplars. We can start to use this to actually learn and start to do better by their students in meaningful ways. Like, I think we're going to start to see some really fascinating and UDL is like universal design of learning. We're going to start to see some fascinating ways in which we're going to start to see learning design really shift and change. I think it's where we are going. Our ultimate goal of this presentation is not that we have this nailed, everyone. I also want to say that we are thinking about this the same time that this is being built. And so we are coming to our own decision making and trying out and testing and seeing what really resonates, what the capabilities of the tools are right now.

Amanda Bickerstaff

We're also really excited about the opportunity to start to look at something like this and actually even just say, hey, what about my assessments that don't work for students? How can I start improving those right now? So what we're going to do is, I think at this stage, Amanda, when this is done, you guys can kind of follow these along. And if you do, I love how Sophia, like, try it out. You want to try it on Claude, you want to try it on copilot or you want to try it on Gemini? It's always fun to do a little bit of a test, actually have the different tools come next to each other.

Amanda Bickerstaff

I would say I would always use something like this, and if it suggested exemplars, I would be using Claude to write the exemplar because I actually think that Claude's a good writer. But there are these opportunities for us to start to think through. How are we, the ultimate goal here is how can we use generative AI itself to actually improve our assessment practices? And that's going to be different. And I think for everyone right now, as you're starting to build your own capacity and also the ways in which you know what you have and what in your student audience and starting to pull that all together. So, Amanda, do you mind putting just going and sharing the piece in the. Yeah, it says at the top right. The very top corner. I would like. This was really interesting, everybody. There's also the opportunity.

Amanda Bickerstaff

Now, I don't know if everybody's seen this, but you could share this. So it's available on the web, which was a really interesting thing that happened when Google Bard first came out, that people's search of people's conversations ended up being indexed and searched. So I would just be a little bit. See, there we go. Make this chat discoverable. I would be a little bit careful also, especially if you're doing something very specific to your practice. So probably don't want to share that publicly. And so. Okay, let's come off the chair, and, like, we have a couple minutes left with everybody. And first of all, I just want to say thank you to everyone that's gotten really kind of evolved. And in the chat, we have so many good questions.

Amanda Bickerstaff

It's clearly everybody going to be something that we're going to do more than one on. So we're going to continue to do work and share from our end other pieces. But I kind of want to wrap us back up into the purpose of the presentation and kind of give a couple of challenges and call to actions for everyone going forward. If you know us, you know, I love, you know, I love a challenge, but I think that this is the opportunity to start thinking through, like, what we can do. And so I'm going to share my screen one more time. And like, when we talked about this piece here of like, the opportunity of process of our product, we saw this like, break this down, including multiple rounds of feedback, self assessment and reflection.

Amanda Bickerstaff

But here are some ways and what I will definitely put this into the follow up materials of this chart, specifically of where we can take an AI vulnerable assessment and create an AI resistant. And so the idea of a multiple choice or true false quiz, an essay, a type lab report or a book report is like the idea of where we can start to do this. So for a book report, students actually annotating a text and then having that conversation, were actually talking about this yesterday about this annotated bibliography that we asked students to kind of create those and they're kind of for us, the teacher, and less for the student being able to actually use that to build a product.

Amanda Bickerstaff

Like could we use this where we kind of put it on where the kids annotating the things they really enjoy and we can actually, essentially, we could even take these, put them into a document, use a generative AI tool to assess them and analyze them and see who's got similarities and differences and actually create groups where students are actively talking about the book instead of it being this kind of passive book report instead of a lab report, students can actually, like, we could actually be about the experiment. This is what I love. I'm a science teacher, everybody, a former science teacher. And lab reports were really hard to get my high school students from the Bronx to do.

Amanda Bickerstaff

But if I said, okay, we're going to actually, you're going to need to show me a video of the parts of the experiment that work that didn't work, and you have to talk through why it worked and didn't work and how you were resilient and what you learned about this. That could be a really fascinating way that's very student focused that still going to be assessing them on how the experiment went, but it's less about a formulaic lab report and more about the learning and the reflection that happened, which is really important for us.

Amanda Bickerstaff

And so this is a good example of, like, how you can start to shift these pieces, and we'll do a lot more on this as we go forward, because there are all kinds of really cool ways to start to unpick these traditional assessments and not just make them AI resistant, but actually make them potentially more meaningful to students, more authentic. It's actually drive at the actual conceptual understanding that you're looking for than what we have done in the past. I'm going to show one more thing. Is that okay, Amanda, I'm actually, guys, I'm going to go a little bit off book, which does happen sometimes. And we have a. In our prompt library, we have a prompt that is actually one of the very first prompts I wrote before Amanda existed on our team. And it's a bloom's taxonomy prompt.

Amanda Bickerstaff

And I kind of have come back to this because bloom's taxonomy is such a rich tool for us in the sense of actually thinking about the stages of learning an application. But we very rarely use this as a way to actually be tactical and practical in classrooms. And so this taxonomy is actually designed. If I go to chat GBT, you can see I was doing it yesterday, guys. I was actually doing this live, but if I go even to the free version, I drop this in and starting to think about priming it.

Amanda Bickerstaff

But I want to start creating a bloom taxonomy for a specific grade and for an actual activity in this conceptual framework, which is studying fractions, I can start to build out these kind of strategies at each level that if I take this, I can start to make sure that I'm actually, as a teacher, moving students along the bloom's taxonomy as if I was doing this for 8th graders. Like, if I was doing this for, like, here we go. I did one yesterday on generative AI, this idea, bloom Saxon on me for 10th graders around AI literacy, because we love a meta moment, everybody. And I was saying, like, what could we do if we read these and we saw that these were all going to be really, like, thoughtful and meaningful? We use our expertise.

Amanda Bickerstaff

We refine them, as always, could I give this to students and say, you're going to pick one or two from each bucket, and that's what you're going to focus on, and you're going to choose your own adventure and your path. Actually showing me how we're going to move towards this level of, like, not just understanding, but that we are truly going to be able to evaluate and create on your own. And so I think that this is where it gets really interesting and it's something which I don't think is possible. And I'll make sure to drop this into the chat right now.

Amanda Bickerstaff

I don't think this was possible to do in terms of, like, before chats, VT and being able to do that, you know, actually starting to think through the different steps that we have and being able to, like, be more intentional about what we're asking and getting that extra support. So, everyone, we're a little bit over time, there's so many questions. I think let's, what we'll do is we'll make a point to be able to try to answer some of these going forward. And I think that what we will do as well is that all the great chat from today, what we'll do is we'll make sure that we're using that going forward in terms of what we do next.

Amanda Bickerstaff

And so we're going to definitely do more of this, but we're going to use everything we learned today from you all the provocations, the pushbacks, the thoughts, the things. We're going to use that. But there's always a way to keep involved with us. We'll be posting our next webinar soon. We've got our prompt library that we're updating. I think you're going to update next week, which is really excited with some new prompts. We've got our free course and then we have just our newsletter. So just feel free to keep hanging out. We love working with you all. I also want to say, can we say thank you to Amanda? This is Amanda's first webinar with us, and she has done so much.

Amanda Bickerstaff

She is the power behind this content, and we just really appreciate everybody and thank you so much for being a part with us today. And we just really hope to see you soon. Have a good morning, night, afternoon, wherever you all, and I hope to see you all soon. Bye, everybody.