Peer Review and Digital Badges paper published

Our paper on the project has now been published. This gives a good overall summary of the project, the rationale for the underpinning design and the analysis of the evaluation project. The paper is published in Chemistry Education Research and Practice, which is free to read (although you may need to register). The paper should be open access in the next few weeks (i.e. no registration necessary). Enjoy!

All resources developed are freely available to reuse as described in the previous post.

developing-laboratory-skills-by-incorporating-peer-review-and-digital-badges

Experiment Done & Resources Available

This week was D-Day…Week; we had our first years completing their lab skills lab. If you want to try what we did, I’ve linked everything below.

Each student had to:

  • prepare in advance for the lab by:
    • completing a pre-lab survey (this is for the associated research project)
    • watch the pre-lab videos;
  • complete the techniques in the lab while
    • being videoed on their phone for two techniques and
    • completing peer review forms for three techniques;
  • after the lab they had to
    • Upload their video to labs afterwards to a video sharing site;
    • Submit the video links for review.

Overall it seemed to go well. There’s quite a lot to unpack, and having spent every spare moment this week reviewing the titration videos, I haven’t much more energy to say a lot beyond that. But I can say while I was expecting chaos and to be running around the lab coralling students, it just ran like any other lab. Students had something to do. They did it.

Feedback takes two forms.

  1. Students get feedback on their technique via the VLE. This involves watching their videos and giving individual feedback. It is very time consuming, but offers a very rare opportunity to provide specific technique feedback to students. I will be able to build up a good assessment checklist now having gone through lots of them, which is exactly what Marcy Towns said would happen when she described her own implementation.
  2. Students also get feedback on their peer review forms. This gives them feedback on number of decimals recorded, calculations, and significant figures. It was done en masse using mail merge and some Excel formulae under the bonnet, which again means you can give some very individualized feedback, but this time quite easily. (Love mail merge!)

Resources

I’ve put the links to the resources we used below. These are freely available to use, without permission, for anyone who wants to use them. I would ask that if you do use them to let me know how you get on – I like to hear the anecdotes. I would again like to acknowledge the IAD for funding and the project team.

Exemplar Videos.

Peer Review Sheets. I’ve added the Word file at this link so you can customize for your own institution/preferences

Digital Badges. These fabulous badges were designed for the project. If you click on them you should get the original high quality PNG file.

 

Final Badges for issuing

We are counting down now to our final days before our students complete their lab badges skills. Just over a week to go! To move things along, we have gotten the badge back from the designer and they look really great! I’m thinking of making these flyers to try to summarise to students what it is they are all about.

titration-badge-flyer

The final videos we are using this year are at:

 

Getting ready to badge and looking for interested partners

Over the summer we have been working on a lab skills badging project. Lots of detail is on this site, but briefly this is what it’s about:

  • Experimental skills are a crucial component of student laboratory learning, but we rarely assess them, or even check them, formally. For schools, there is a requirement to show that students are doing practical work.
  • By implementing a system whereby students review particular lab techniques in advance of labs, demonstrate them to a peer while being videod, reviews the technique with a peer using a checklist, and uploads the video for assessment, we intend that students will be able to learn and perform the technique to a high standard.
  • The video can form part of students electronic portfolio that they may wish to share in future (See this article for more on that).
  • The process is suitable for digital badging – awarding of an electronic badge acknowledging competency in a particular skill (think scout badges for… tying knots…).

Marcy Towns has a nice paper on this for pipetting and we are going to trial it for this and some other lab techniques.

Looking for interested parties to trial it out

I am looking for school teachers who would like to try this method out. It can be used to document any lab technique or procedure you like. You don’t necessarily need an exemplar video, but a core requirement is that you want to document students laboratory work formally, and acknowledge achievement in this work by a digital badge. We will provide the means to offer the badge, and exemplar videos if you need them, assuming they are within our stock. Interested teachers will be responsible for local implementation and assessment of quality (i.e. making the call on whether a badge is issued).

Yes I need help with badge design
Yes I need help with badge design

This will be part of a larger project and there will be some research on the value and impact of the digital badges, drawing from implementation case studies. This will be discussed with individuals, depending on their own local circumstances.

So if you are interested, let’s badge! You can contact me at: michael.seery@ed.ac.uk to follow up.

Badging Workshop and Titration exemplar

I attended a useful workshop given by Marcy Towns and colleagues of Purdue at the BCCE 2016 conference. They published the paper on badging for pipette techniques. A few things came out of the workshop:

  1. Student videos were less than 2 minutes long. A series of common errors were noted in pipetting.
    • This study didn’t use exemplar videos, but did use demonstrations in the lab, so it will be interesting to see if exemplars reduce common errors in the first instance.
    • This has prompted me to think about assessment – we will need to have prompts to focus students on demonstrating the actual technique and just recording that.
    • Therefore we will need to tell them not to video the washing procedures, and probably point them to what they must demonstrate. Towns used this to develop a rubric subsequently.
  2. Silent videos are scary.
    • In their first iteration, students duly demonstrated what they were meant to, but didn’t narrate. Students are now told to narrate, to ensure that it is clear that they know what they are doing in each step and why they are doing it.
  3. When to do it.
    • The Purdue implementation allowed students to make the videos at any stage of the lab course. In our case, we are dedicating an early (Week 2) lab session to it, as other labs are very busy and we feel that some students may not have the time to complete the videoing. We also want them to get the idea of good technique early.
  4. It worked!
    • By several measures, including costs of broken glassware, tutor and demonstrator observations, and students reporting themselves, the process of demonstration worked. I hope that by adding a peer-review step, we will emphasise the notion of correct technique by adding in an interim review step before videos are uploaded.

We are on the last leg now. I hope that this is the final iteration of the titration video. I decided to cut the pipetting from it, and have it as a parallel video:
Titration Exemplar: https://media.ed.ac.uk/media/Titration+Exemplar+Video/1_wmnz31b5

End of Internship

Hello again! I’m now back home as my internship ended last Thursday. In this post, I’ll try to summarize what I managed to do and what I learnt from this experience.

The first technique was titration. I started from finding sources and reading them. Then I made the protocol and started filming. The main difficulty was to find out how to obtain the best quality of the video. I published the first version and got some really nice feedback. I went back to the lab and made the second version, and now finished the post production – so the final video is waiting to be published. In the meantime, high school students came to the Uni to give us feedback on the project.

The second video was standard solutions. This was much easier to do – firstly because the technique itself was a bit shorter, secondly because I had more experience after the first video.

The third video – a technique that took me the most time – was distillation. I’ve only recently published the first version for feedback.

I also managed to write protocols for a few more techniques and video one of them – filtration.

So, what did I learn from the internship? Firstly, I practiced the techniques. Secondly, how you can never be sure if you know something until you start teaching it. Teaching requires being proficient at things, so you’ll notice fast if you’re not good enough at something. Moreover, at first I thought that three videos in seven weeks is a joke and I could easily do even 20. Only when I started making videos, did I realize how wrong I had been.

So, to sum up, I really enjoyed the internship – it was a nice challenge and a very good work experience, no matter what career path I will choose after Uni.

I would like to thank some people for making this easier for me to do what I managed to. Firstly, I’d like to thank Dr Michael Seery for giving me the opportunity and for introducing me to the project. Secondly, to Ms Elaine Sutherland and Ms Kirsty Bain for invaluable help in the lab. To Dr Murray Low, Dr Robin Vevers and Dr Kristy Turner for huge help with the scientific part of the project, and to Euan, Amy and Helen for being great help as well.

I will, of course, follow the project and will see you in my fifth year! 🙂