Peer Marking

Today's Toronto Star has an interesting article on Peer Marking Gets a Negative Grade.

Students in one of our first year psychology classes were asked to submit a short writing assignment to an online evaluation program called "peerScholar." The site, which was developed by teachers at our Scarborough campus, is set up to allow papers to be graded anonomously by fellow students.

University of Toronto Teaching Assistants, represented by their union (CUPE local 3902), objected. They say this is a blatent attempt to do away with TA's in favor of a computer program. The students in the class also expressed some concern, according to the newspaper article. Apparently, students see the peer evaluation system as "inaccurate" and "unfair."

I first heard about this experiment last year when I went to a presentation by one of the authors. What impressed me was the possibility of teaching students how to critically evaluate the work of their fellow students. I was also interested in giving students some direct experience in how their grades are determined. There's no better way to learn how the system works than grading a fellow student. We did it when I was in school. As a matter of fact, I took a university course where our entire grade was based on a group discussion at the end of term where we assigned grades for each other, by consensus. But that was the 60's.

The peerScholar project seemed like a good way of introducing more participation into a course and I was/am seriously thinking of using it in my course. Here's a desrciption of how it works in the PSYA01 class. There's more information on the peerScholar discussion page. I think it's unifortunate that the authors put so much emphasis on saving money by avoiding TA's. While I recognize that's a legitimate concern, I think that peer evaluation is an important goal by itself.

I've seen the data on fairness and accuracy and it's very impressive. Students tend to be a little too hard on their colleagues but that's easy to compensate for. By the second evaluation they've become much better. If the practice were more common, the students would get much better at it. As it is, the grade assigned by the students is at least as good as that assigned by a TA. It tends to deviate more from the grade assigned by the Professors, but then so do the TA's grades.