Interview as Exam: Part 4 - Assessment Tools
This is the fourth post in a series on the Interview as Exam. The first outlines why we did it: the causes and consequences for me and my students of doing this kind of assessment. The second explains the first kind of interview we started with in our history department. The third explains how and why we simplified the interview and supported the learning all the way through the course. This post is about assessing the interview.
We agreed as a teacher team that we should have at least two records of the interview. The student brings their own notes and the teacher should take their own notes and/or audio record the interview in addition. Impartiality was much less of an issue than we imagined it might be since we had clear questions and we were clear that our role in the interview was to listen, not talk. If we wanted to really see what they could do, and assess our students fairly, we had to not put words in their mouths. If you look at a transcript from my students' interviews, you can see this fairly clearly. As my students are speaking, I'm really listening, and, for me, taking notes helps me to listen. For other teachers, note-taking was a distraction, so they felt more comfortable with just the audio recording. When you are really listening, you "catch" a lot more than when you're halfway through a stack of essays or exams. The student is there in front of you. You have a personal and professional obligation to honour their time and their effort in sharing their understanding with you.
In the first iteration of the interview, we had students put the 5 Ws of the event into the middle of the organizer. Even though we said we wouldn't ask them about it, they couldn't resist going back to it, and often this took up so much time in the interview that we didn't have enough time to get into the actual thinking and argumentation. So, for the second iteration, we took out the 5 Ws. We were able to assess their historical knowledge about events by having them explain their thinking about historical events, plus it was a lot less boring! Having 15 students in a row explain to me what happened at Vimy Ridge, or how women got the right to vote, or any other event is not really my idea of a good time. I'm much more interested in their analysis of these events and in hearing how they can apply the historical thinking concepts to help shape their understanding of their own experiences in the present.
Warning: Learning in Progress!
Whenever people ask to see my rubrics or assessment tools for the interview, I have an internal cringe. It's not that I don't like to share (see my teaching wiki or Twitter!) it's that the formal assessment process was really still in process when I left the classroom to work as an instructional coach in 2015. A lot of the initial assessment was kind of "loose." Things did get tightened up over the years, but I've learned a lot in the past few years that I would now apply a little differently in the classroom, like using better success criteria, for example.
I am also never sure what people are really asking when they ask for my assessment tools. Do they want a rubric, or do they have questions like I did when we started:
I am also never sure what people are really asking when they ask for my assessment tools. Do they want a rubric, or do they have questions like I did when we started:
- Will I be able to be impartial?
- Will I catch everything that the student is trying to say?
- Is 10 minutes too long? Too short?
- Should I take notes, or listen? Audio, video record and transcribe?
- What if a student contests their result? How will I defend my position?
- Will students be able to plagiarize? How will I know if they REALLY know it?
- Will some students be disadvantaged?
- Are they still getting enough practice at writing?
- How much more work will it be?
- Taking notes, recording, transcribing? Will we be able to be impartial? Will we "catch" everything?
We agreed as a teacher team that we should have at least two records of the interview. The student brings their own notes and the teacher should take their own notes and/or audio record the interview in addition. Impartiality was much less of an issue than we imagined it might be since we had clear questions and we were clear that our role in the interview was to listen, not talk. If we wanted to really see what they could do, and assess our students fairly, we had to not put words in their mouths. If you look at a transcript from my students' interviews, you can see this fairly clearly. As my students are speaking, I'm really listening, and, for me, taking notes helps me to listen. For other teachers, note-taking was a distraction, so they felt more comfortable with just the audio recording. When you are really listening, you "catch" a lot more than when you're halfway through a stack of essays or exams. The student is there in front of you. You have a personal and professional obligation to honour their time and their effort in sharing their understanding with you.
- Is a 10-minute interview too short? too long?
In the first iteration of the interview, we had students put the 5 Ws of the event into the middle of the organizer. Even though we said we wouldn't ask them about it, they couldn't resist going back to it, and often this took up so much time in the interview that we didn't have enough time to get into the actual thinking and argumentation. So, for the second iteration, we took out the 5 Ws. We were able to assess their historical knowledge about events by having them explain their thinking about historical events, plus it was a lot less boring! Having 15 students in a row explain to me what happened at Vimy Ridge, or how women got the right to vote, or any other event is not really my idea of a good time. I'm much more interested in their analysis of these events and in hearing how they can apply the historical thinking concepts to help shape their understanding of their own experiences in the present.
- What if a student contests their result? How will I defend my position?
While I never actually had a student contest the result of an interview, I felt confident that I had two or three sources of evidence to review or defend a result. I had the student's own notes, my own notes, and an audio file that could be reviewed if necessary. If I had time, I would often give the student their result (in a level) on the spot. There are ways of making audio tangible, and technology makes it easier almost daily. There are tons of apps on a Chromebook, or Google Read-Write or lots of other ways make audio recording super-accessible to almost anyone.
- Will students be able to plagiarize? How will I know if they REALLY know it?
Of course, people can lie in an interview, but my experience was that my students were actually much more honest with me than I had been anticipating. In the first iteration, we had students pull a primary source "out of a hat" (from the set of 20 sources we had given them in advance) and they had to analyze and explain it on the spot. One of my students got this great Prohibition cartoon that I had found on the Begbie Contest Society site. He was struggling to explain it, so much so that I just asked him: "Do you know what a "bootlegger" is?" and he just said: "No, miss, I don't." in the saddest voice I had ever heard. So, I explained the word, and he was able to then do a little bit of analysis of the cartoon. He didn't get top marks for knowledge, but he was able to demonstrate at least some thinking skills. Truly, knowing the word "bootlegger" is not that important for your life, but knowing how to decode visual information, and knowing how to contextualize and analyze it is a critical skill for all learners.
- Will some students be disadvantaged?
The first time we did the interviews, the students who were most anxious about it, were the students who typically did very well academically. They all did more or less the same (very well) on the interview as they would have on a written piece or exam. We reflected on that anxiety and made sure to give students at least 1 or 2 other opportunities to do an interview in the semester. That helped reduce their anxiety a lot.
The students who made the most gains were students who were already struggling: those who had a learning disability, or ADHD or some other challenge. They were able to clearly articulate their ideas at their intellectual/academic level. That was so exciting for us! That we could help the kids who were struggling the most to be able to meet or exceed their potential was really exciting and garnered us a lot of support from the Spec Ed department and our administrators pretty quickly.
The students who made the most gains were students who were already struggling: those who had a learning disability, or ADHD or some other challenge. They were able to clearly articulate their ideas at their intellectual/academic level. That was so exciting for us! That we could help the kids who were struggling the most to be able to meet or exceed their potential was really exciting and garnered us a lot of support from the Spec Ed department and our administrators pretty quickly.
- Are they still getting enough practice at writing?
Yes! Students did have to write quite a lot in the rest of the course - in a number of styles from essay outlines, to historical fiction and newspaper articles. They had another final project that was a writing or media product, so we did have lots of opportunities to help support students in developing their writing skills. What was really fantastic was that by taking away the writing requirement, we could then really see their thinking and help support them in developing the building blocks of good critical writing - analysis, and evidence, which can then transfer to all kinds of other writing forms, including essays.
- How much more work will it be?
Okay, we didn't actually have this question. We actually thought it would be much less work than setting and marking class sets of poorly written exams. We gleefully scheduled a full day (minus lunch break) of 10-minute interviews on our assigned exam days. By the end of that first day, I was EXHAUSTED! Listening is really hard work!
I decided the next semester to only do a morning of interviews on the exam day, which was much more manageable and fairer to the students as well. The rest of the students I interviewed in the last few days of class, at lunch or after school, or even in the afternoons during the exam period. That way, each student would get my full attention, and I could really hone in on what they had learned. Other teachers found other solutions that worked well for them. Since we didn't worry whether students would see the questions ahead of time, the asynchronicity was actually liberating.
I decided the next semester to only do a morning of interviews on the exam day, which was much more manageable and fairer to the students as well. The rest of the students I interviewed in the last few days of class, at lunch or after school, or even in the afternoons during the exam period. That way, each student would get my full attention, and I could really hone in on what they had learned. Other teachers found other solutions that worked well for them. Since we didn't worry whether students would see the questions ahead of time, the asynchronicity was actually liberating.
After a few years, we realized that, while we didn't have the stacks of marking, it was just as much work as the old-school exams, but it was different, and, amazingly the work was JOYFUL! I found myself really looking forward to the interviews - What did my students learn this year? What did they get excited about? What will I learn from them? It really did become a celebration of learning. Not every interview was amazing, but every one of them gave me insight into my students' learning and helped me to plan to support their learning better next time.
The Tools
I developed a number of different assessment tools to go with the Grade 10 History course (CHC2D/P). In Ontario, this is Canada from 1914-Present.
CHC2D (Academic) - this tool has a formal rubric using the Ontario curriculum expectations.
CHC2P (Applied) - this tool includes a more generic list of success criteria. I developed the descriptors.
CHC2P (Applied) - this tool includes a more generic list of success criteria. I developed the descriptors.
Other courses that I have used interviews:
CHY4U - World History 1500-Present, grade 12, University destination. This tool used the same generic success criteria, but the class helped to develop some specific descriptors.
CHW3M - World History to 1500, grade 11, University/College (Mixed). This is just the question and outline for the students. I just took notes and used the criteria outlined in the previous examples. What really was interesting with this question, was that most students figured out that mistakes and achievements are highly dependent on your perspective, and most of them ended up explaining how that worked for each event.
In all of these courses, I also asked students to reflect on the skills of history that they learned, and how they learned them. I had them reflect throughout the course on these skills and record their thoughts at the end of each unit in the My Skills Inventory (discussed in more detail in Part 3). While it was a bit more challenging to tie these directly to specific expectations, it was invaluable in helping students to take ownership of their own learning, and it gave them one more opportunity to share their learning with me, which was really my favourite part of the interview.
In all of these courses, I also asked students to reflect on the skills of history that they learned, and how they learned them. I had them reflect throughout the course on these skills and record their thoughts at the end of each unit in the My Skills Inventory (discussed in more detail in Part 3). While it was a bit more challenging to tie these directly to specific expectations, it was invaluable in helping students to take ownership of their own learning, and it gave them one more opportunity to share their learning with me, which was really my favourite part of the interview.
Comments
Post a Comment