Tools in Piloting Phase

New learning technologies at UBC undergo a rigorous process of evaluation and preparation before rolling out to the wider community as an available and supported tool. These are a few of the innovative projects currently being developed and assessed.

Jump to tool: ComPAIR | Course Spaces | Crowdmark | Mattermost


ComPAIR

New Application

CTLT has built a brand-new application for peer feedback called ComPAIR. ComPAIR was conceived as a way of using the Adaptive Comparative Judgement (ACJ) method as part of student assignments. This method is based on the law of comparative judgement (L.L. Thurstone, 1927), which holds that people are far more reliable when comparing one thing with another than when making absolute judgements without any comparison.

How does it work?

Beginning of the answer pair comparison in the application

Beginning of the answer pair comparison in the application

In the application, students first answer questions posed by instructors and then compare and comment on three or more pairs of peer answers presented side-by-side. From these comparisons, a score is calculated for each answer according to ACJ’s method, and a ranked list is presented to instructors that should roughly reflect the “best quality” to “lowest quality” answers to the question.

The process helps students practice critical-thinking, peer feedback, and self-reflection, while instructors and teaching assistants get a pre-sorted list of answers that can provide quick examples of strong/weak responses and potentially expedite grading. (Instructors may also remove the ranking feature and use the tool solely for peer feedback purposes.)

What’s next?

The tool has gone through the initial piloting and evaluation stage (using courses in English, Math, and Physics), and students, teaching assistants, instructors, and CTLT researchers have provided valuable feedback. Further work is planned on the application to refine its algorithm and functionality as well as integrate it into the UBC ecosystem before it will be available to all instructors.

If you are interested in using the application during this beta stage, you can contact the LT Hub or download and install the open-source code from Github.


Course Spaces

New Platform

CTLT is working on a new WordPress-based learning platform to provide a course delivery option between the functionality of a full LMS (Learning Management System) and a course blog/website. Spaces will provide a good alternative for instructors who find Connect too bulky and UBC Blogs too lightweight.

The primary goals with this new platform are:

  • Simplify the online learning experience without losing the requisite functionality
  • Make a consistent, fast, mobile-first interface that displays content in a highly accessible way
  • Fix many of the common problems experienced by students and instructors with existing tools
  • Make open content easy to manage and access

How does it work?

Similar to UBC Blogs, instructors will set up a course website (a “space”) and manage it from a user-friendly administrator interface, publishing content for students to interact with and viewing student work and interactions.

What’s next?

The development team has sought lots of input from faculty, staff, and students through informal conversations and presentations of the current design and concept. A prototype for a deeper dive hands-on experimentation will be released in the near future for limited pilot use.


Crowdmark

New Application

The Crowdmark application interface

How marking works in the application

Crowdmark is an online grading and analytics platform. Crowdmark supports exam grading by managing online distribution of exam papers to graders; facilitating online grading and feedback; automating paper return to students; and providing examination analytics.

How does it work?

Information on the general Crowdmark workflow is available at crowdmark.com. However, since Crowdmark is currently hosted on servers outside of Canada, there are additional privacy considerations for using the service in a manner that is compliant with BC’s Freedom of Information and Protection of Privacy Act (FIPPA). See below for details.

Managing Privacy of Student information

For general information on student privacy, see the LT Hub copyright privacy page and additional links on that page. In particular, the fact sheet Disclosing Personal Information Outside Canada provides guidelines on using cloud services in compliant ways.

There are two basic approaches to using Crowdmark in a FIPPA-compliant way:

Option 1: Ask for student consent

Inform students that the service for grading exams is hosted on servers outside of Canada, and ask the students to provide consent for using the service. Students should be given the option to opt-out of the service if they have concerns. Students who opt-out would have their exams marked offline.

The following boilerplate language may be used to inform students:
In this course, an online grading tool, Crowdmark, will be used to support the examination grading process. Crowdmark is an online grading solution that operates on servers outside Canada. By agreeing to this service, you are consenting to the storage of your personal details (name, email and grade) outside of Canada. While Crowdmark adheres to strict privacy regulations, UBC cannot guarantee security of your private details on servers outside of Canada. If you choose not to provide your consent, see your instructor to make alternate arrangements.

Option 2: Tokenize (hide) student identity
Establish what is known as a “tokenization” process to hide student identities, so that personally identifiable information is not ever stored outside Canada on Crowdmark servers. A locally-stored and secured copy of the identity-mapping table would need to be maintained to facilitate this process; this table is how you associate anonymous students in Crowdmark to actual students in your course. Note that this approach presents a few logistical challenges, including allocating the exam papers to the correct students during the examination period, as well as returning the exam after grading.

If you have additional questions about supporting these different approaches or about Crowdmark generally, you can email James Colliander directly.

What happens next?

The tool is currently being piloted in a number of courses during the 2016 W1 term. It is expected that an evaluation will be completed by the end of the year.


Mattermost

New Application

The Faculty of Education, in collaboration with the Learning Technology Hub, is currently piloting Mattermost, an open-source chat tool that facilitates communication and collaboration.

Mattermost, a chat and communications tool

A look at the Mattermost interface

Mattermost allows for the creation of course “teams”, making online spaces where classmates can share messages and files. Public channels, private groups, and 1-1 direct messages can be used to keep conversations relevant.

Real-time and asynchronous communication are combined, meaning quick flurries of messages and threaded discussions can coexist in an “anytime” communication environment. All messages are persistent and can be searched, so nobody misses out.

How does it work?

Mattermost can be accessed via the web, and there are desktop and mobile apps for most platforms. Email, desktop, and push notifications can be set up to personalize the experience.

The tool is open source and hosted at UBC. Access to a Mattermost team can be granted through the use of email invitations or an invitation URL.

What’s next?

Mattermost was selected as a chat tool after an environmental scan and evaluation that considered pedagogical, technological, usability, and accessibility requirements.

If you think Mattermost could be effective in your course and would like to try it out, please contact the LT Hub to schedule a hands-on orientation. Your feedback is especially valuable as this tool is considered for wider use at UBC.