Following my presentation yesterday I wanted to expland on my experiences of testing the two peer marking tools available to us at Leicester University (I will report on my use of google presenter + chat used in the presentation in another blog post).
Technical notes: we are using Blackboard 9.0 with Turnitin fully integrated in to the VLE. The Peer and Self Assessment tool is the same as was included in Bb 8.0, following development by Dundee University.
1. Set up
Blackboard Peer and Self Assessment Tool: Fairly straight forward to set up, with pretty good notes explaining what all the terms mean. If you think in terms of the student work being submitted as short answer questions, each of which can be peer assessed under a set of criteria, then set-up is straight forward.
PeerMark: I found setting this up very confusing, the interface has very little support or instruction and it is not at all clear what is set where. I always seem to get an error asking for latest submission dates to be set when I haven’t chosen late submissions to be allowed. Perhaps this is related to the VLE integration, but it is very confusing. There are several places where you need to click on ‘more details’ or a a further link which enables you to add what I thought was essential information rather than ‘optional’ extra information. See video.
2. Student use
Testing these types of assignments is pretty hard to do on your own, you need at least two peers to help you. Unless work has been submitted to assignment you can’t see the peer review element, and unless peer reviews have been submitted you can’t see how the results would look. Blackboard wins here as it has a ‘preview’ feature which does help you to geel confident about how instructions will appear to the students.
Several colleagues commented that the Blackboard submission and review process was very ‘click’ heavy (which it is) and very difficult to know what to do when. When accessing the evaulation section it is not at all obvious how to open the work that needs to be peer assessed and several colleagues didn’t realise they had work to review.
In comparison, PeerMark is well thought out from teh student user point of view. Having the peer work and criteria on one page makes the process simple to complete. A status bar at the top of the page shows when the review is 100% complete and can be submitted.
One colleague commented that finding the feedback from the peer reviews was very difficult in PeerMark, buried under several layers.
3. Instructor evaluation of the results
Blackboard wins out slightly here, as marks given by peers were easily seen on a results list, and submissions and evaluations were easy to access. PeerMark was more cumbersome and not as easy to see which students had submitted work and what their comments were.
4. Features and application
I had compare the two services in my presentation:
PeerMark was much more flexible in terms of the ways that peers could be allocated to review other student work and in a striaght comparison would win hands-down if it weren’t for the awful set-up interface and confusing results section (for staff and students). As it is, it’s probably a tie and either tool could be used depending on the task at hand.
I made a series of videos to demonstrate staff and student views of PeerMark and the Blackboard tool, essentially because I couldn’t trust the tech in my presentation, but also for using with instructors as neither tool is particularly easy to explain!
Setting up PeerMark (2:23)
Student reviewing work in PeerMark (2:28)
Instructor view of PeerMark results and evaluations (0:45)
Blackboard Peer and Self Assessment
Setting up a Peer Assessment in Blackboard (3:50)
Student submission of work to Blackboard peer assessment tool (0:53)
Peer review by student of Blackboard assignment (1:38)
Instructor view of Blackboard peer review tool results, evaluations and student submissions (1:54)