ComPAIR Instructor Guide

Plan your student peer assessments

As with any learning activity, how these assignments are introduced, designed, and integrated in the course impacts the student experience with ComPAIR. Here are some recommendations to consider.

• Clearly explain the learning objective(s): Students who understand the underlying skills they are practicing in ComPAIR feel they learn more from the assignments. It is important to explain the end goal of using ComPAIR—how comparing will help learning—rather than merely explaining the process. As a starting point, download the modifiable template for introducing ComPAIR to students created by UBC instructors.
• Build on ComPAIR assignments: ComPAIR assignments are more beneficial to students when presented as part of a larger process. It is better to use ComPAIR for peer review in the context of a bigger course goal (e.g., prep for writing a later term paper or evaluation of an early project draft), rather than for standalone assignments.
• Provide a low-stakes training round: Students feel more confident when they can practice comparing answers in a low-stakes training round prior to graded ComPAIR assignments. These practice rounds are especially useful in conjunction with a review of how to conduct comparisons.
• Use multiple, detailed criteria for comparisons: When given more guidance, students say they learn more from comparing, especially when assignments are complex. Explaining precisely what to look for during comparisons—with multiple descriptive criteria in the application and/or detailed external rubrics—may result in stronger learning outcomes and richer feedback.
• Require 3+ comparisons per assignment: Students uncertain about giving peer feedback report stronger confidence in this skill after practicing with more answer pairs. A minimum of three comparisons per student (i.e., each student reviews six peer works) is suggested to allow students to practice.
• Assess the quality of peer feedback: Adding weight to the peer feedback that students write helps instructors increase the quality of feedback given. Assigning marks creates additional accountability for students and increases their commitment to the feedback process.

Develop your student peer assessment criteria

In ComPAIR, one or more custom criteria can be used to guide students' comparison of the answer pairs. Each criterion must pose a question that can be answered by choosing between the answers (e.g., Which is better?).

To develop criteria, start with the course learning outcomes and think about how seeing other work will help students get there—what can students learn from each other? Keep in mind how experienced students in the course will be and use criteria at a complexity the majority will be able to understand. UBC instructors generally use between 1-6 total criteria and advise using an odd number to discourage students from selecting a “tie” between answers.

Additionally, consider if the goal is more summative (critically looking at final work) or formative (constructively looking at drafted work). Harder, more complex, and longer criteria may be better for summative uses. Easier, more straightforward, and shorter criteria may be better for formative uses.

Decide on the timing of your assignment phases

You will need to decide how many ComPAIR assignments you will give in the course and the timing for each phase of the assignment: answering, comparing, and (optionally) self-evaluating. By default, comparing begins after answering ends, though these may be designed to overlap. However, keeping the phases separate allows students to complete their evaluations smoothly, without the delays that can happen when answering and comparing overlap and there are not enough answers available for students to compare.

ComPAIR assignments can be given during and after group projects as well. When you use the group assignment feature, only one answer will be required from each group (any member can submit), but each group member must complete the required comparisons and self-evaluation (if enabled) individually.

Choose how outcomes will influence grades

If ComPAIR assignments will influence grades, communicate to students from the beginning exactly how the assessment marks will be counted.

Often, instructors grade with a combination of two aspects:

• Students' participation in the process (i.e., answering, comparing, and/or self-evaluating) - When creating your ComPAIR assignment, you will be able to assign weight for completing each phase.
• The quality of the work students do through ComPAIR - This quality can apply to any of the following: the answers students initially submit, how peer feedback informs a revised or later version of the submitted work, or the thoroughness and thoughtfulness of their peer feedback (which has the added benefit of increasing students’ sense of accountability in providing quality constructive criticism).

A note on ComPAIR's rankings

ComPAIR’s focus is to help students effectively review peer work and write insightful feedback using comparison. However, some instructors have considered using ComPAIR’s optional scoring/ranking of answers for grading as well, primarily by chunking the answers into top, middle, and bottom groups and then assigning a grade to each group.

Conduct your own validation if you intend to factor ComPAIR’s outcomes into the marks you assign. Internal research indicates that ComPAIR’s scoring may not reliably map to traditional expert grading, as it relies on the skills and training of novices (students) to provide accurate, well-informed rankings of their peers’ answers.

Set up a ComPAIR course

The recommended way to get started with ComPAIR is to set up a ComPAIR course through Canvas. Students will be added when they click the ComPAIR link in your Canvas course. The setup instructions differ depending on whether you want to sync the ComPAIR participation grades with the Canvas Gradebook.

Understand the ComPAIR workflow

Using ComPAIR for peer review is a multi-stage process.

1. You create an assignment: You will configure five options for each assignment: the assignment details, timing of phases, comparison criteria, number of comparisons, and participation grade weights.
2. Students answer the assignment: Students answer using a rich text editor, a file attachment, or both. For a group assignment, only one student from each group needs to answer.
3. Students compare and comment on pairs of peer answers: Students select a preferred answer from the pair for each criterion you set (e.g., "Which is better?), then write feedback to the author of each answer, following a set of customizable instructions. This phase is done individually, even in group assignments.
4. Students complete a self-evaluation (optional): If you enable self-evaluation, students reflect and comment on their own answer at the end of the process, following a set of customizable instructions. This phase is done individually, even in a group assignments.
5. Everyone views results: Once the phases of a peer review assignment are complete, students access the peer feedback given to their (or their group’s) answer, as well as all answers submitted by peers. You and your teaching team can read and comment privately or publicly on student answers, access a participation table to view activity and participation grades, mark "top" answers, and view each student’s comparison choices, peer feedback, and self-evaluation.

To find out more about each peer-review step, visit the ComPAIR support site's workflow page.

Create an assignment in your ComPAIR course

1. Log in to ComPAIR using your UBC CWL and click your course name. If you do not have a course, you can follow the previous steps to set one up.
2. Click +Add Assignment and fill in information about the assignment:
   • Under "Assignment Details", set the short name of the assignment, enable/disable self-evaluation and group functionality, and add assignment instructions in "Additional Assignment Information". You also have the option to upload a related attachment.
   • Under "Schedule", set the timelines for when students can answer, compare answers, and (optionally) evaluate their own answer. By default, the start date for comparing is after answering ends and there is no end date for comparing or completing self-evaluations. To add a start and/or end date for these phases, check the Manually set when students compare answers box and/or the Manually set when students self-evaluate box.
   • Under "Prompts", set the criteria that students will use to compare the answer pairs. Click +Add New to use more than one criterion. Each criterion should pose a question answered by choosing between the answers. Instructions for peer feedback and self-evaluation can also be added and customized.
   • Under "Comparison Settings", set the number of comparisons required from each student and how the pairs will be selected by the application—at random or adaptively based on how students have ranked answers so far. You can also add a practice answer pair for students to start with, enable replies to let students leave optional additional feedback, and allow instructors and TAs to compare.
   • Under "Participation Grade", set the weight given for completing each part of the assignment. These weights will be used to calculate a total participation grade.
3. Click Save.

Set up grade syncing with Canvas for a ComPAIR assignment

Once you have created a ComPAIR course and assignment, you can set up grade syncing with Canvas.

To sync properly, you will need to set up individual links in Canvas to each ComPAIR assignment. Students must click the link of each Canvas assignment to establish a connection between their Canvas Gradebook entries and their work in ComPAIR.

If you want to manually enter grades instead of syncing participation grades from ComPAIR to Canvas, these steps are not required.

1. To create an assignment link in Canvas, first copy the necessary information from ComPAIR:
   • Log in to ComPAIR using your UBC CWL and click your course title.
   • Scroll to the ComPAIR assignment you want to sync grades from and click Edit underneath the assignment title.
   • Click Copy? at the top right of the edit page. This action will copy the information you need to enter in Canvas.
2. Log in to your Canvas course and click Assignments in the Course Navigation.
3. Click +Assignment in the upper right side of the screen.
4. Enter the assignment name as you want students to see it.
5. Scroll down to "Submission Type" and select External Tool from the drop-down.
6. To ensure the assignment will sync properly with Canvas, click Find instead of entering the URL directly.
7. From the list, pick ComPAIR and click Select.
8. At the end of the "External Tool URL" field, type a question mark (?), then paste the value you copied from ComPAIR.
   • The "External Tool URL" will look similar to this:
     compair.elearning.ubc.ca/api/lti/auth?assignment=*******
9. Check the Load This Tool In A New Tab box. This setting allows grades to properly sync with Canvas.
10. Click Save & Publish so students can access the assignment.
11. Make sure to tell your students that they need to click each ComPAIR assignment link in Canvas for syncing to work; otherwise, they will not receive the grades they expect.