Badging Workshop and Titration exemplar

I attended a useful workshop given by Marcy Towns and colleagues of Purdue at the BCCE 2016 conference. They published the paper on badging for pipette techniques. A few things came out of the workshop:

  1. Student videos were less than 2 minutes long. A series of common errors were noted in pipetting.
    • This study didn’t use exemplar videos, but did use demonstrations in the lab, so it will be interesting to see if exemplars reduce common errors in the first instance.
    • This has prompted me to think about assessment – we will need to have prompts to focus students on demonstrating the actual technique and just recording that.
    • Therefore we will need to tell them not to video the washing procedures, and probably point them to what they must demonstrate. Towns used this to develop a rubric subsequently.
  2. Silent videos are scary.
    • In their first iteration, students duly demonstrated what they were meant to, but didn’t narrate. Students are now told to narrate, to ensure that it is clear that they know what they are doing in each step and why they are doing it.
  3. When to do it.
    • The Purdue implementation allowed students to make the videos at any stage of the lab course. In our case, we are dedicating an early (Week 2) lab session to it, as other labs are very busy and we feel that some students may not have the time to complete the videoing. We also want them to get the idea of good technique early.
  4. It worked!
    • By several measures, including costs of broken glassware, tutor and demonstrator observations, and students reporting themselves, the process of demonstration worked. I hope that by adding a peer-review step, we will emphasise the notion of correct technique by adding in an interim review step before videos are uploaded.

We are on the last leg now. I hope that this is the final iteration of the titration video. I decided to cut the pipetting from it, and have it as a parallel video:
Titration Exemplar: https://media.ed.ac.uk/media/Titration+Exemplar+Video/1_wmnz31b5

What it looks like to issue and receive a badge

The second prototype of the badging platform built on experience described last week; namely that peer-review will need to take place in advance of assignment submission, and that a tutor should look at submissions to make sure they are what they say they are before a badge is issued.

With that in mind, a new prototype involves:

  1. Students watching and clicking to confirm that they have watched the exemplar video.
  2. Students uploading their video to obtain a particular badge.
  3. Tutors reviewing video and awarding a score. It is possible to set a minimum score necessary before badge is issued.

So what does this look like? We will look at the tutor view as well as what the student sees.

As a tutor, I can add the exemplar videos and create an assignment dropbox for each technique. Then, as outlined in the last post, I can use the achievement tool to create triggers that (1) require that students review a video (and click to say they have reviewed); (2) to upload an assignment file; and (3) to have the file reviewed by tutor before badge is issued. In the case of the latter, a minimum mark can be set which the student has to receive before the badge issue is triggered. As a rough outline, the screen below shows the rudiments of this from the student view:

info sheet 1

Trialling the process

I logged in as a student, watched the video (or at least clicked to say I watched it) and uploaded a file. The good news is that a video file upload is possible, although whether this is what is done is going to be the subject of another post. As a student, I have done all that is required of me.

Back in tutor view, I see in the gradebook that there is an assignment that needs marking.

assignment

I can view this and award a grade (100/100 – I am a great student!). This was the final trigger for badge issue, and so the badge is issued for this student. We can see the number of recipients for this achievement increases to ‘1’ in the Achievement control area (number on the right).

achievements with one recipient

The student view, and exploring Mozilla Backpack

Back as a student, I now see that I have one new achievement in the Achievements area.

Badge Issue

There is an option now to publish to Mozilla Backpack, so that I can share my badge with the world. Clicking on the option connects me with Mozilla – this first time I need to sign up. After confirming, my “Badge Issue Trial” badge is now in my Backpack. This tells me the name of the issuer, and details of what was required to get the badge. I can add this badge to my “collections”, and after making it public, share via social media and other channels.

Mozilla

Building the platform… and the 1st compromise

The technical side of the project involves having a platform where the exemplar videos will be available for student review, where the students will submit their video, and complete the peer review. Once these three are completed, it will automagically trigger the issuing of a badge, which is immediately published to Mozilla’s Open Badges platform, so that the student can link to the badges in their “Back Pack” at any point in the future.

I’ve been planning to use Coursesites, which is Blackboard’s free version of.. well… Blackboard, so that I can create a mirror for my own in-house students but also make the system available to anyone who wants to use it.

coursesites

Blackboard (and hence Coursesites) were a little late to the badging party, but they now offer the option of what they call “Achievements”. As an instructor, I can set a certain number of things the student has to do before earning a badge. Once the student completes these, the badge is issued, and (after toggling on in the settings) pushed to the student’s own Open Badges Back Pack. The student can see the available achievements, which in this case will correspond to particular experimental techniques. 

achievement home

So far so good. I created three requirements in this test. The first was to watch a video – the student marks it as “Reviewed”. The second was to upload a video (file) as an assignment. The third was to introduce peer assessment so that students could review their peer’s video. 

achievements - requirements

Unexpected item in the Badging Area…

And there comes the snag. Blackboard in all of its wonderfulness only enables peer assessment of text material. While students can submit any file they like for an instructor to correct, they may only look at text based work from each other. So that’s that! Now you could argue that this is something I should have checked before writing a grant proposal, but that wouldn’t be nice… Moodle has something called Workshops, but… Moodle…

A compromise…

I want peer review to be part of the process. So if it can’t be done in the submission stage, it means we will have to do it pre-submission. So gather around people – rub that out, here’s a new plan;

  1. Students review exemplar videos [Achievement Part 1 – unlocked!].
  2. Students demo it to each other in the lab. In turns, the peer reviews the live demo or the review on the mobile phone using checklist provided.
  3. Students re-record if necessary after peer review.
  4. Students upload it to review platform [Achievement Part 2 – unlocked!]
  5. Demonstrator watches submitted video and scores to same rubric [Achievement Part 3 – unlocked!].
  6. Student is awarded a badge.

This compromise mightn’t be all that bad. It;s moving the peer review earlier in the process so it happens in real time. In fact it might be better than the original. Why?

  • It means that peer review is happening at a time when the peer can do something about it; and there is no doubt students will want to submit the best possible attempt.
  • It means there is less fiddling required by the student on the platform – they just use it now to review the exemplar(s) and submit their video(s).
  • It introduces a tutor correction step; this is something the Towns’ paper included. It was something I was planning to avoid, mostly due to workload issues. But I always had a niggling thought that a student could upload a video of their granny washing the dishes after dinner, review another video and get a badge from the University of Edinburgh saying they could pipette properly. We probably need this step. But it means introducing restrictions on video length, for the sake of the demonstrators correcting. Those wishing to adopt it for their own purposes can opt to toggle this off.

The next attempt will be to build a prototype around this revised model.