Can students evaluate teachers? (part 1 of 2)

I’ve been looking at the results from our annual large-scale survey to our 5500 families. As you would expect, there are a range of opinions – and it can be hard to know how to interpret the conflicting ‘please do more X’ and ‘please do less X‘ comments.

One theme that emerged very clearly, however, was that it’s teachers who make the difference to students, and we were grateful for the parents who told us about the immensely positive transformations experiences that their children are undergoing with inspirational teachers. But there were also many comments that asked us to formally involve students in teacher appraisal and evaluation – sometimes with the implication that there’s a lot of stuff going on that school leadership does not know about.

In a busy organisation it’s impossible for anyone person to know everything that’s happening – but we do have to try.  And I guess the notion is that students are the ones receiving the teaching, so shouldn’t they get a say?

There are two points here.  Firstly, there is the moral need to treat students with respect, and to hear their perspectives about how things are going.  Secondly, there is the analytic need to know how to interpret data that comes from students (given that getting this second point wrong will damage student education, that’s also close to being a moral need too).

In this post I want to suggest that what we know about the second point should directly inform the first point; and that we should not simply ask students to evaluate teachers.  To prevent this post from being too long, I’ll follow up next week with some details about how, despite this, we still can and should involve students.

So why is student evaluation of teachers a bad idea? What we know is that using students to evaluate teachers can be very problematic for many reasons – often because there is confusion between a teacher’s popularity (he’s a nice or funny or charming person) and a teacher’s effectiveness (students learn a lot).  The two are not entirely independent of course, but there is a clear distinction.  We should therefore be troubled by, for example, Uttl et al’s 2016 analysis that shows the correlation between students’ opinions of their teachers and what they  actually learn is close to zero. If this is right, then it’s a central flaw because it means we can learn nothing reliable from the evaluations.   But it is at least better than Rodin and Rodin’s classic paper which suggests that it might even be worse than that – and that in fact students rate most highly the teachers from whom they learn least, and furthermore that students don’t learn more from teachers with high ratings – so the evaluations may actually be inverted! Perhaps many students will like a teacher who goes easy on a class and gives high grades for mediocre work, thus leading students to feel good about themselves and consequently give high ratings to that teacher.  The teacher who sticks to high standards, gives tough love and honest un-inflated grades, by contrast, will likely not be so popular, and may receive lower ratings.  Rodin and Rodin speculate ‘perhaps students resent instructors who force them to work too hard and to learn more than they would wish’

This is not surprising to experienced teachers who can distinguish between the long- and short-term effect of what they do, based on years of experience.  These teachers worry that if the evaluations are high-stakes, there will be systemic pressure on them to be popular rather than effective.  I think we can all see where that has led to if we look at the global political stage.  Nothing serious should be decided solely on popularity.

A further reason to doubt student capacity to judge teachers is that judgement like this is subject to serious bias.  Hessler et al found that teachers who provided chocolate cookies during a course got significantly higher evaluations than those that did not.  No surprise there either!  Other biases are more serious.  MacNell et al undertook an ingenious online study, where teachers told one class they were male, and one class they were female.  They found that ‘students gave professors they thought were male much higher evaluations across the board than they did professors they thought were female, regardless of what gender the
professors actually were. When they told students they were men, both the male and female professors got a bump in ratings. When they told the students they were women, they took a hit in ratings. Because everything else was the same about them, this difference has to be the result of gender bias’.

So we know that student evaluations are problematic.  So what?  Let me return to and affirm the first point – that there is a moral need to involve students; and to hear their voices.  So how can we do it? 

It is possible to do; but it involves different approach, away from an evaluative consumer model to a collaborative one.  I’ll write about that in some detail next week.

Addendum
Just came across this – 

Carrell, S. E and West, J.E (2010) Does Professor Quality Matter? Evidence fromRandom Assignment of Students to Professors 

Excerpts:
We compare metrics that capture these three different notions of instructional quality and present evidence that professors who excel at promoting contemporaneous student achievement teach in ways that improve their student evaluations but harm the follow-on achievement of their students in more advanced class..