ComPAIR Instructor Guide

Plan your student peer assessments

As with any learning activity, how these assignments are introduced, designed, and integrated in the course impacts the student experience with ComPAIR. Here are some recommendations to consider.

• Clearly explain the learning objective(s): Students who understand what underlying skills they are practicing in the application feel they learn more from the assignments. It is important to explain the end goal of using ComPAIR—how comparing will concretely help learning—rather than only explain the process. As a starting point, download the modifiable template for introducing ComPAIR to students that UBC instructors have created.
• Build on ComPAIR assignments: Assignments in ComPAIR are more beneficial to students when presented as part of a larger process. It is better to use ComPAIR for peer review in the context of a bigger course goal (e.g., prep for writing a later term paper, evaluation of an early project draft), rather than for standalone assignments or “busy work”.
• Provide a low-stakes training round: Students feel more confident when they can practice comparing answers in a low-stakes training round, prior to any graded ComPAIR assignments. These practice rounds are especially useful when a review of how to conduct comparisons is included afterwards.
• Use multiple, detailed criteria for comparisons: When given more guidance, students say they learn more from comparing, especially when assignments are more complex. Explaining precisely what to look for during comparisons—with multiple descriptive criteria in the application and/or detailed external rubrics that are provided and explained ahead of time—may result in stronger learning and possibly richer feedback.
• Require 3+ comparisons per assignment: Students uncertain about giving peer feedback report stronger confidence in this skill after practicing with more answer pairs. A minimum of three comparisons per student (i.e., each student reviews six peer works) is suggested to give students repeated practice to learn from.
• Assess the quality of peer feedback: Adding weight to the peer feedback students write helps instructors increase the quality of feedback given. Assigning marks creates additional accountability for students and increases their commitment to the process.

Develop your student peer assessment criteria

In ComPAIR, one or more custom criteria can be used to guide students' comparison of the answer pairs. Each criterion must pose a question answered by choosing between the answers (e.g., Which is better?).

To develop criteria, start with the course learning outcomes and think about how seeing other work will help students get there—what can students learn from each other? Keep in mind how experienced students in the course will be, and use criteria at a complexity that the majority will be able to understand. UBC instructors generally stick with between 1-6 total criteria, and some advise using an odd number to discourage students from selecting a “tie” between answers.

Consider also if the goal is more summative (critically looking at final work) or formative (constructively looking at drafted work). Harder, more complex, and longer criteria may be better for summative uses. Easier, more straightforward, and shorter criteria may be better for formative uses.

Decide on the timing of your assignment phases

You will need to decide how many ComPAIR assignments you will give in the course as well as the timing for each phase of the assignment: answering, comparing, and (optionally) self-evaluating. By default, comparing begins after answering ends, though these may be designed to overlap. However, keeping the phases separate allows students to complete their evaluations smoothly (without the delays that can happen if answering and comparing overlap and there aren't enough answers available for students to compare).

ComPAIR assignments can be given during and after group projects as well. When you use the group assignment feature, only one answer will be required from each group (which any member can submit), but each group member must complete the required comparisons and self-evaluation (if enabled) individually.

Choose how outcomes will influence grades

If ComPAIR assignments will influence grades, you should communicate to students from the beginning how exactly the assessment marks will be counted, so students are not caught off-guard.

Often, instructors grade students with a combination of two aspects:

• Their participation in the process (i.e., answering, comparing, and/or self-evaluating). When creating your ComPAIR assignment, you will be able to assign the weight given for completing each phase of the assignment.
• The quality of the work students do through ComPAIR, which may mean any of the following: reviewing the answers students initially submit, seeing how peer feedback informs a revised or later work they submit, or looking at the thoroughness and thoughtfulness of their peer feedback (which has the added benefit of increasing students’ sense of accountability in providing quality constructive criticism).

A note on ComPAIR's rankings

ComPAIR’s main pedagogical focus is to help students effectively review peer work and write insightful feedback using the power of comparison. However, some instructors have considered using ComPAIR’s optional scoring/ranking of answers for grading as well, primarily by chunking the answers into rough top, middle, and bottom groups and assigning a grade to each.

Conduct your own validation, if you intend to factor ComPAIR’s outcomes into the marks you assign. Internal research so far indicates that ComPAIR’s scoring may not reliably map to traditional expert grading, as it ultimately relies on the skills and training of novices (students) to provide accurate, well-informed rankings of their peers’ answers.

Set up a ComPAIR course

The easiest way to get started with ComPAIR is in Canvas. Through Canvas, you can set up a ComPAIR course, and students will be added to it once they click the ComPAIR link in Canvas. The instructions differ depending on whether you want to sync the ComPAIR participation grades with the Canvas Gradebook or not.

Understand the ComPAIR workflow

Using ComPAIR for peer review is a multi-stage process.

1. You create an assignment: You need to configure five options (the assignment details, timing of phases, comparison criteria, number of comparisons, and participation grade) for each assignment, a process that can be completed in under ten minutes.
2. Students answer the assignment: Students answer using a rich text editor, a file attachment, or both. For a group assignment, only one student from each group needs to answer.
3. Students compare and comment on pairs of peer answers: Students select a preferred answer from the pair for each criterion you set (e.g., "Which is better?), then write feedback to the author of each answer, following instructions that you can customize. This phase is done individually, whether or not it is a group assignment.
4. Students complete a self-evaluation (optional): If you enable self-evaluation, students reflect and comment on their own answer at the end of the process, following instructions that you can customize. This phase is done individually, whether or not it is a group assignment.
5. Everyone views results: Once the phases of a peer review assignment are complete, students access the peer feedback given to their (or their group’s) answer, as well as all answers submitted by peers. You and your teaching team can read and comment privately or publicly on student answers, access a participation table to view activity and participation grades, mark "top" answers, and view each student’s comparison choices, peer feedback, and self-evaluation.

To find out more about each step, visit the ComPAIR support site's workflow page.

Create an assignment in your ComPAIR course

1. Log in to ComPAIR, and click your course name. If you don't have a course, you can follow the earlier steps to set one up.
2. Click Add Assignment, which will take you to a page for filling in information about the assignment:
   • Under "Assignment Details", set the short name of the assignment, enable/disable self-evaluation and group functionality, and add any assignment instructions.
   • Under "Schedule", set the timelines for when students can answer, compare answers and (optionally) evaluate their own answer.
   • Under "Prompts", set the criteria that students will use to compare the answer pairs. One or more criteria can be used and each must pose a question answered by choosing between the answers. Instructions for peer feedback and self-evaluation can also be customized.
   • Under "Comparison Settings", set the number of comparisons required from each student and how the pairs will be selected by the application—at random or adaptively based on how students have ranked answers so far. You can also add a practice answer pair for students to start with and enable replies.
   • Under "Participation Grade", set the weight given for completing each part of the assignment, which is used in calculating a total participation grade.
3. Save the assignment.

Set up grade syncing with Canvas for a ComPAIR assignment

Once you have a ComPAIR course and assignment, you can set up grade syncing with Canvas.

To sync properly, you will need to set up individual links in Canvas to each ComPAIR assignment. Each student must click the link of each Canvas assignment to establish a connection between their Canvas Gradebook entries and their work in ComPAIR.

1. To create an assignment link in Canvas, first copy the necessary information from ComPAIR:
   • Log in to ComPAIR, and click your course title.
   • Scroll to the ComPAIR assignment you want to sync grades from and click Edit underneath the assignment title.
   • Click Copy? at the top right of the edit assignment page. This will copy the information you need to enter in Canvas.
2. Log in to your Canvas course, and click Assignments in the Course Navigation.
3. Click +Assignment in the upper right side of the screen.
4. Enter the assignment name as you want students to see it.
5. Scroll down to submission type and select External Tool from the drop-down menu.
6. Even if you know the URL, click Find (entering the URL directly will not sync properly with Canvas).
7. From the list, pick ComPAIR and click Select.
8. At the end of the "External Tool URL" field, type a question mark (?), then paste the value you copied from ComPAIR.
   • The "External Tool URL" will ultimately look something like this:
     compair.elearning.ubc.ca/api/lti/auth?assignment=*******
9. Check Load This Tool In A New Tab. This allows grades to properly sync with Canvas.
10. Save & Publish the assignment (students can only access published assignments).
11. Make sure to inform your students that they have to click each ComPAIR assignment link in Canvas for syncing to work, otherwise they won't receive the grades they expect.