Research and Engagement

In the first year of the Learning Analytics project, research and engagement activities centre primarily around engaging faculty in the development and piloting of learning analytics projects focused on improving student learning and success. These pilots are aimed at identifying questions that faculty have, exploring actionable sources of data, and trialing candidate tools that empower faculty members and departments with analytics to address the questions they have identified. At a high level, the overarching focus of the Learning Analytics project’s research and engagement efforts is to facilitate a community of faculty members who are actively engaged in the development of learning analytics models and student learning research.


Engagement Strategy

The Learning Analytics project engagement strategy is being carried out through collaborations between UBC’s Learning Analytics working group and faculty members, primarily through one-on-one consultations and through the research pilots outlined on this page. The project’s engagement goals include the following:

  • Engaging with faculty to identify key questions that can be answered using analytics.
  • Empowering faculty members and departments with analytics to address challenges they have identified at the course or program level.
  • Evaluating the appropriateness and usefulness of various approaches to interacting with and displaying information.
  • Working with faculty to identify relevant data.
  • Facilitating a community of faculty members who are actively engaged in the development of learning analytics models and student learning research.

Learning Analytics Workshops

UBC faculty and instructional support staff are invited to learn more about Canvas Analytics at an upcoming hands-on workshop. Participants will learn how to use Canvas Analytics tools and will explore ways in which analytics can be used to support student engagement and refine teaching practices. Participants will also have opportunities to learn more about new and emerging analytics tools and to share feedback about possible features that could make learning analytics even more useful.

Canvas analytics provides instructors with information about learner activity, assignment submissions, and more.

For instructional support staff:
Exploring How Canvas Analytics Can Be Used to Improve Teaching and Learning Outcomes
12:15 p.m. – 1:15 pm
Tuesday, February 27, 2018

For instructors:
Exploring How Canvas Analytics Can Be Used to Improve Teaching and Learning Outcomes
1:00 p.m. – 2:00 p.m.
Thursday, March 8, 2018

 

 

 

 


Research Pilots

VizIT, a learning analytics tool for edX that’s being explored through a research pilot at UBC, can help instructors stay better informed about learner activity. This visualization  highlights problems students in a course had the most difficulty with.

Research pilots, which will be active from September 2017–August 2018, are a central component of the Learning Analytics project. The areas of focus for the pilots were determined based on faculty submissions to UBC’s call for Learning Analytics proposals issued in June 2017. The following criteria were taken into account when reviewing faculty submissions: potential benefits to students; anticipated impact beyond the initial project; feasibility in terms of data availability and complexity; and balance across the portfolio of projects.

More information about the Learning Analytics pilots was livestreamed on June 13, 2017. Watch the full video.

There is considerable proficiency in learning analytics at UBC, and the research pilots outlined below build on existing community-based strengths in terms of expertise and interest. Pilot goals include building traction in learning analytics, facilitating the development of prototypes, and gaining insight into uncharted areas to help guide UBC’s overall approach to learning analytics.

Each of the research pilots is part of an ongoing, iterative exploration process, and our understanding of community interest and the achievability of pilot objectives will continue to evolve. As a result, some degree of fluidity can be expected in terms of the pilot descriptions outlined below.

Pilot Topic: Analytics for Teaching

An instructor-facing dashboard for Canvas that displays student activity

UBC instructors who teach large blended and online courses have requested tools that will help them keep better track of students’ online activity and performance. This pilot seeks to identify possible enhancements to Canvas’s activity dashboard that could provide instructors with improved insight into student activity.

VizIT: An instructor-facing dashboard for edX that shows student activity

VizIT, a tool for edX, provides instructors with a comprehensive view of students’ online course activity data. This tool can be used with both edX Edge-based UBC courses and edX-based UBC MOOCs (massive open online courses). In this pilot, UBC faculty members using edX and the Learning Analytics Working Group are seeking to evaluate the effectiveness of the VizIT course dashboard by providing instructors with data about the types and levels of student engagement with course material.

OnTask: Providing timely, personalized, and actionable feedback to learners

Giving timely, relevant feedback to students is a critical aspect of good teaching and supporting student success. This pilot gives instructors the opportunity to try OnTask, a web-based tool that uses data from online learning tools such as Canvas to allow instructors to define progress indicators, track students’ progress, and give targeted feedback based on the metrics they set for their course. OnTask includes a dashboard showing individual and aggregated student progress metrics and allows instructors to easily send custom email or text feedback to subsets of students who have yet to meet performance indicators, giving them a personalized reminder of what they need to do to catch up.

Strategies and data requirements for effective student team formation

This project investigates ways of visualizing student group characteristics and supporting team formation based on student profile information. Ultimately investigators are interested in understanding how team composition affects team performance and how to optimize group formation for different learning outcomes. This project may also lead to the development of a tool that suggests teams based on instructors’ preferred team composition strategies.

Threadz: Network analysis and visualization of Canvas discussions

It can be a challenge for instructors to effectively monitor online class discussions. This pilot explores the Threadz network tool, which integrates with Canvas to provide instructors with visualizations of what's happening in course discussions and reliable metrics for measuring student engagement. Developed by Eastern Washington University, this tool enables instructors to easily access data about who is engaged, whose responses are going unanswered, and whether students are achieving the engagement goals that were set for the course. Threadz can also alert instructors to potential issues early on, such as learner isolation and instructor-centric discussions, which can provide insights that help make discussions more helpful for all learners.


Pilot Topic: Analytics for Learning

A student-facing dashboard for Canvas that displays anonymous, comparative activity data

Providing students with data about their own online course activity and engagement with learning materials, compared with the activity of anonymized peers, has been demonstrated to support self-monitoring and self-directed learning. The goal of this pilot is to field test a student-facing activity tool embedded in Canvas that provides students with up-to-date data on their own activity versus that of their peers.

A student-facing dashboard for edX that provides personalised feedback based on previously successful learners' activity

Compared with students who are taking an in-person courses, students in a fully online course lack information about how and when their peers who are successful in the course are engaging with learning materials. To bridge this gap, this pilot seeks to explore a personalized feedback system for students in edX that facilitates anonymized social comparison with previously successful learners. The goal is to offer an interactive, student-facing dashboard that displays visualizations of multiple behavioral indicators. This tool could potentially be implemented for all UBC’s edX MOOCs.

 

Pilot Topic: Analytics for Program Planning and Advising

Data-driven curriculum analysis

This pilot explores statistical and visual techniques for analysing and displaying historic student performance data. Areas of focus for this pilot include course difficulty, the impact of a particular course on overall academic performance, curriculum coherence, dropout paths, and the impact of course load on student performance. This project could lead to the development of models that aid curriculum committees and department heads in decision making regarding course and curriculum offerings. Student advisors could use findings obtained through these analyses when advising students about course difficulty and course load. This information could also be shared with course instructors to provide insight into students’ experiences in their courses or programs.

Predicting success in UBC program specializations to inform admissions criteria

Due to high demand, student admissions into specializations at UBC can be competitive and are often based on factors such as students’ overall GPA. This pilot will explore methods for determining the predictors of success in program specializations. The findings from this pilot could lead to new knowledge that supports better, evidence-based admissions standards that take into account relevant criteria that more effectively predict student success. This pilot may also explore predictors of success in specific courses—for example, by considering helpful or unnecessary prerequisites.

Registration dashboard showing real-time enrolments

Departments and programs face a serious challenge when planning how many course sections to offer and how many instructors to appoint or assign, especially for required courses. Patterns of student registration are complex and highly dynamic, and it’s difficult for departments to obtain useful data or to project future enrolments. Underestimation results in overfull classes—and students being unable to register in necessary courses—while overestimation can result in half-full sections and inefficient use of resources. This pilot seeks to develop predictive models, based on past enrolment, with the goal of projecting final enrolment numbers, including final student count after add/drop cut-off dates. Ultimately, this project could help programs and departments allocate resources more effectively and support better course planning, contributing to student satisfaction and success by ensuring more students are able to get into required and preferred courses.

Student success dashboards

At UBC, the Student Information System (SIS) collects data on enrolment pathways, grades, failure rates, graduation rates, and other variables. This pilot explores ways of using this data to develop faculty- or department-level diagnostic dashboards that provide an overview of whether and how students may be struggling. These types of data could ultimately be made available to all department heads and senior leadership, supporting evidence-based decision making that improves student learning and success.

Visualizing and reporting on student enrolment pathways to inform curriculum review

Departments and curriculum committees have increasingly asked for help in understanding how students move through their courses and programs. Requests have been made for overviews that will give departments a ‘big picture’ understanding of common enrolment pathways and sequences or pathway differences between groups (e.g., majors vs. non-majors). Other requests are focused and seek to answer specific questions about the impact of a particular course or the effect of changing program requirements or course offerings. This pilot could lead to insights into actual patterns of student enrolments, which could be beneficial for instructors, departments, and curriculum committees. These insights could also be used to advise students who are considering various possible paths through their degrees.

 

Call for Pilot Participants

Faculty members who are interested in participating in any of the learning analytics research pilots outlined above or who would like more information are invited to contact Ido Roll (Senior Manager for Research and Evaluation, Centre for Teaching, Learning and Technology).