I designed an ineffective assessment last week. Here’s what happened:
My mentor teacher and I wanted to find out (or, my mentor teacher instructed me to find out) whether our 6th grade Language Arts students were ready to move on from the five elements of plot. Had we spent sufficient time here before moving on to characterization and setting? Or did we need to spend more time reviewing the basic structure of plot? Did students need more practice identifying these in short stories?
So I set out to design an assessment involving the five elements of plot. The simplest format would have been to have students diagram the plot of a story we read in class. But this felt predictable, since we’ve been diagramming the plots of stories for a few days. If I want to see how effectively they can outline a plot, I can look in their composition books at the work I’ve already asked them to do. I wanted this assessment to show me something I didn’t already know. What were my other options… Of course I could have just asked them to regurgitate the definitions of the five elements of plot (you know, exposition, rising action, climax, falling action, resolution), but that wouldn’t have generated the type of data I was after. I wasn’t trying to figure out if they’d learned the terms, I wanted to see if they could identify the concepts in real literature. An assessment that asked for the definitions would have been too easy.
“Too easy” – that’s part of the problem. I’m unreasonably afraid of creating something that’s too easy. So now my goal had changed. Not only was I trying to figure out how well they understood the elements of plot, I was also aiming to design an assessment that was “difficult enough.”
There’s no such thing as “difficult enough.”
I thought what I had to do was make it harder. I didn’t like that it was easy. I thought – the way to find out if they really know what I want them to know is to make it harder. A harder assessment will prove they that can use what they know. But I was wrong about that.
Making an assessment harder doesn’t make kids think harder, and it doesn’t give me more accurate picture of what they can do. If I want an accurate picture of what they can do, I need to have a clear idea of what I want them to learn before I start to teach it, then make sure my lessons are aligned with that objective, then make sure my assessments are appropriately aligned as well.
A smarter, more experienced person put it this way:
To construct an individual test, begin by visualizing the performance you expect from your students. Next, create specific objectives, using Bloom’s taxonomy… to make sure you address different levels of knowledge and understanding. From these objectives, construct a test blueprint, and then write individual items to address each of your objectives and topics (Shermis, 2001).
The beauty of Bloom’s taxonomy is that is rescues us from the misunderstanding that, in order to reveal important data, a test must be difficult. Difficulty should certainly be a consideration in creating assessments, but the chief concern should be matching the level of thinking (or the language function) among learning target, practice task, and test item. That’s how a teacher collects accurate data on what students have or haven’t learned.
The super embarrassing part of all this is that I was supposed to have already known that. I think I did know it, but I didn’t know how to look for it. Maybe I will now.
What I finally decided to do was to ask them to identify and defend their identification (hopefully using some piece of the relevant definition) of five different moments in the story. Also, I would scramble the different pieces from the story, so that their responses to my questions couldn’t just follow the same pattern as the diagram at the top of the page. I made it harder than diagramming a story, see? Because I’m so tough.
And then I decided that since my assessment was so tough, I’d start with a blank plot diagram at the top of the page. I’d ask students to label the five elements on this diagram before moving on to the super tough scrambling stuff I’d ask later. This was a brilliant idea that I had, both because it would allow most of them to feel successful before they go to the gloriously difficult questions, and because it’d perhaps provide insight into why some of my students would get some of the questions wrong. I was very impressed with myself.
The assessment was too difficult. The disassembled story fragments were difficult to place out of order, let alone match with the appropriate vocabulary term. My supervisor (did I mention that all of this was for the day my supervisor came to observe me for the first time?) said afterward, “I’ll bet that many of these kids would have been able to accurately label the elements of that story if they had seen them in order, or if they could have seen them all at once.” I considered both of these alternatives when I planned the assessment, and discarded both because they were “too easy.” But level of difficulty isn’t important until I’ve consulted Bloom on the level of thinking. I’ll remember that for next time.
 No one ever made a movie about a teacher whose assessments were easy.