peerScholar Instructor Guide

Plan your student peer assessments

As with any learning activity, how these assignments are introduced, designed, and integrated in courses impacts the student experience with peerScholar. A UBC pilot and related evaluation of peerScholar highlighted some general recommendations to consider in your planning.

• Explain the learning objective(s) to students: Outline the end goal of using peerScholar in your course—why student peer assessment can help with learning and why peerScholar has been chosen for doing the activity.
• Clearly articulate peerScholar’s phases and process to students: The workflow can be complicated, especially if you include extra steps like the feedback-on-feedback, self-assessment, and/or revision. Make sure students understand what will be involved in the assignment and when each step needs to be completed.
• Train students to use contextual commenting: peerScholar lets students directly annotate each other's assignments. This contextual feedback can be helpful for learning, as it encourages granular engagement with peer work. However, the feature can be a bit hidden, unless students are trained in how to use it.
• Enable and incentivize the feedback-on-feedback feature: peerScholar allows students to evaluate the feedback they receive. This feature is another reported perk of the system, both from student and instructor perspectives. It closes the feedback loop, allowing students to learn what makes peer feedback more or less useful. Also adding marks for how deeply students engage with these feedback evaluations can motivate students to leave more detailed feedback-on-feedback.
• Prepare to deal with marking variations: Not all students will complete all phases of the assignments, so be prepared for how to grade around these drop-offs. For example, if you will use the marks that students receive on their feedback as part of grading, consider how you will accommodate a student whose feedback was only marked by one student.
• Run internal test assignments before going live: Test assignments completed by the teaching team are a good way to identify if what you want to do with the tool is possible and practical.

You will also want to think about how many student peer assessments each student should complete. Keep in mind how much time it will take to properly review the type of work that you assigned.

Develop your student peer assessment rubrics

In peerScholar, you can set up virtually any kind of assessment, customizing both the questions and question types that students respond to in their assignments. You will set the questions for the assessing phase (where students review one another's work) as well as the optional self-assessment phase (where students review their own work) and the feedback evaluation phase (where students review the peer feedback they've received). Questions can be assigned relative weights, depending on how important each criterion is to that phase of the assignment.

To develop your rubrics, start with the course learning outcomes and think about how evaluations will help students get there—what can students learn from each other? Keep in mind how experienced students in the course will be, and use rubrics at a difficulty level that the majority will be able to understand. Requiring detailed written peer feedback as part of the evaluation also helps students think more deeply about the peer work that they review.

Note that peerScholar does not incorporate a training phase. You can instead plan an in-class activity before the first assignment. In this activity, discuss the rubrics with students and model or have students practice applying the rubrics to peer work.

Decide on the timing of your assignment phases

You will need to decide how many peerScholar assignments you will give in the course, as well as the timing for each phase of the assignment: creating, assessing, and reflecting. In deciding on the right amount, consider the time commitment required from students to complete the phases as you've designed them. It may be difficult to fit more than three assignments per term, but this number will depend on your context.

Choose how outcomes will influence grades

For each peerScholar assignment, you will be able to choose how marks are generated, meaning how much impact your marks, the student peer marks, and the participation marks have on each student's final grade. Once you've decided on distribution, you should communicate to students in the beginning how exactly marks will be generated, so they are not surprised later.

If you choose to use the marks given by students in the final grade, also ask yourself the following:

• How will you examine peer assessments to ensure fairness and accuracy? When marks are on the line, students may sometimes try to influence them with exaggerated reviews (good and bad). How will you investigate outliers, and what will you do with outliers?
• What grading calculations will best fit for your course? Typically, peer marks count for a smaller percentage of the assignment.
• How will you deal with marking variations? Not all students will complete all phases of the assignments, so you'll need to know how you'll grade around these drop-offs.

Understand the peerScholar workflow

peerScholar asks students to perform student peer assessment in three sequential phases:

1. Create - Many instructors have students submit a written assignment for peer assessment. But submissions can be in other mediums, including images and other multimedia.
2. Assess - Each peer work is presented anonymously for students to analyze, mark, and provide feedback on, all based on the custom rubrics you set up when building the assignment. Students can also be required to assess their own work as part of this phase, using the same or different custom rubrics.
3. Reflect - Students read and reflect on the peer feedback that they received. Depending on how you set up the assignment, students may be asked to evaluate each piece of feedback they received and/or implement feedback suggestions by submitting a revised version of the original work.

To learn more about peerScholar's assignment phases and the underlying purpose of each, you can read the peerScholar vision document.

Set up a peerScholar assignment

Through Canvas, you can set up a peerScholar assignment that will be open to all students enrolled in your Canvas course. Assignments in peerScholar are called activities, but these terms are interchangeable.

1. Log in to your Canvas course, and click Assignments in the Course Navigation.
2. Click the +Assignment blue button in the upper right side of the screen.
3. Enter the assignment name, e.g., "peerScholar Assignment #1".
4. Scroll down to submission type and select External Tool from the drop-down menu.
5. Even if you know the URL, click Find. (Entering the URL directly will not sync properly with Canvas.)
6. From the list, pick peerScholar and click Select.
7. Check Load This Tool In A New Tab.
8. Scroll down to click Save.
   • If students already have access to your course, don't publish this assignment yet. You will want to complete setup first.
9. You will be redirected to the assignment page you just saved. Click to load the assignment in a new window to complete building your assignment in peerScholar.
10. You will be prompted to create a new activity or copy an existing one. If you are creating a new one, select the activity type:
    • Classic Individual - An individual student peer assessment assignment.
    • Case Study - A conditional student peer assessment assignment, where different students see different cases or sets of instructions to respond to.
    • Team & Group Work - A group student peer assessment assignment.
11. If prompted, enter a name for the assignment, then click Create Activity and follow the prompts to finish setting up the assignment.
12. Once your assignment is created, you will be taken to an overview page. Make the assignment visible to students in peerScholar by clicking the toggle for "Show Activity" so it turns blue.
13. Go back to your Canvas course, and navigate to the page for this assignment (if it is not open already). Click Edit Assignment Settings and scroll down to click Save & Publish to make the assignment visible to students.
14. A new column in your Canvas Gradebook will automatically be created to collect grades. However, you will still need to manually send grades from peerScholar to your Canvas course using the steps outlined in the accordion on grading below.

Monitor student progress on a peerScholar assignment

peerScholar provides detailed information about student progress and grades for each assignment.

1. Log in to your Canvas course, and click Assignments in the Course Navigation.
2. Click the peerScholar assignment that you want to review, then click to load it in a new window.
3. In the peerScholar interface, you will see an overview of the assignment, along with how many students have completed each phase and how many have been graded.
4. Click the Student Progress & Grading tab at the top of the page to monitor student progress more closely.
5. Click the Analytics tab for a visual overview of the completion rate and grade distribution.
6. Under "Activities" in the left-hand sidebar, click any assignment name to switch to another assignment overview, if you have multiple assignments to review in peerScholar.

Grade a peerScholar assignment and send grades to your Canvas course

peerScholar allows you to manage grades and then sync those grades with the Canvas Gradebook.

1. Log in to your Canvas course, and click Assignments in the Course Navigation.
2. Click the peerScholar assignment that you want to grade, then click to load it in a new window.
3. On the right side of the peerScholar assignment overview, you will see a grading box. Click Grade Now to open the grading interface.
4. The name at the top will identify which student you are grading. Below, you will see the student's submission (original and, if included, revised), self-assessment (if included), and feedback received. Switch between peer reviewers by clicking the people icons in the bar below the student's name or switch to feedback the student gave by clicking View Given Feedback in the same bar.
5. Enter your grade and feedback in the grading area on the right, then click Save.
6. At the top of the page, you can advance to the next student by clicking the arrow or using the drop-down.
7. Once you are done grading, click Back at the top left to return to the assignment overview.
8. To sync the grade changes you've made with the Canvas Gradebook, click Gradebook in the left-hand sidebar.
9. Click Sync Grades at the top right, above the grading table.
10. Select the sync method:
    • Smart Sync - Send only new or updated grades in peerScholar to the Canvas Gradebook.
    • All Sync - Send all grades from peerScholar to the Canvas Gradebook.
11. Click Continue and a success message will appear at the bottom of the screen when the sync is complete. Although the sync request will be sent right away, it may take time to show up in the Canvas Gradebook.