LTI Integration + Caliper MediaEvent Metric Profile
This paper describes a collaboration between Blackboard and Kaltura to integrate and capture learning analytics. The information provided will be useful for the Higher Education and K-12 communities, in both technical and non-technical roles, who are interested in plug-and-play interoperability and enterprise-wide educational analytics.
Thanks to the ease of integration offered by Learning Tools Interoperability® (LTI®), students in richly designed online courses may cross many product and vendor boundaries in a single learning session. While the LMS vendor provides a set of data visualizations for learning activity prior to LTI launch, and each LTI application vendor may provide their own activity data visualizations within their tool, teachers need to review analytics across tools to understand the full picture of their student’s learning activity. By combining LTI and Caliper Analytics™ (Caliper), Blackboard accepts and relays student activities from other tool providers such as Kaltura, removing a barrier to understanding the full picture.
With the release of Caliper 1.0 by IMS Global, Blackboard and Kaltura undertook a joint proof of concept, to take Kaltura’s existing LTI integration with Blackboard Learn and the Kaltura framework’s existing event telemetry, and add Caliper MediaEvent support, so that teachers could see combined activity data from Blackboard Learn and Kaltura in one set of reports within Blackboard Learn.
As a result of this work, customers operating Blackboard Learn SaaS deployments with the latest version of the Kaltura Building Block will automatically receive this new teacher-facing Student Activity visualization in Learn without any additional cost or technical work required. This visualization follows the same interaction pattern as other Student Activity visualizations for tests and assignments in the LMS but shows activity data that has been sent via Caliper from the Kaltura service (Fig. 4).
With the current state of LTI integration, teachers had the ability to run Blackboard Learn reports (Fig. 1) that showed student activity within a course up to the launch moment that they opened the Kaltura service but no further, picking up again if and when they returned to the LMS. The teacher could also connect to the Kaltura service and run Kaltura reports (Fig. 3) that showed student activity from the launch moment on; a time-consuming and cumbersome process.
Figure 1. Example of activity data visualization from Blackboard Learn.
Blackboard and Kaltura already captured the relevant student activity events in proprietary format, and of course an LTI integration (without Caliper) already existed between the two services. As it happened, Blackboard was also in the process of enabling Learn to be an event “sensor” using Caliper, which was IMS Conformance Certified for version 1.0 in October 2015. Kaltura was also in the process of getting their integration working against the Learn SaaS deployment option.
This combination made a joint proof of concept very sensible. The team started to work together and set a goal to minimize the difference between existing capabilities and a compliant integration, while prioritizing this work with other work on the individual product roadmaps for the development teams.
Blackboard’s time investment was about 10 developer-weeks, folded into other project work such as the visualization interface development and supporting the Kaltura team. Collaboration was generally by email, and a few live collaboration discussions were held to set up a shared development and testing environment. Kaltura’s investment was approximately 12 developer-hours over the same time period, primarily because existing proprietary instrumentation was in place and updating to Caliper was essentially adopting a packaging standard.
Figure 2. Process Workflow.
To begin sending Caliper MediaEvent events (play, pause, finish) to the same Blackboard endpoint that the LMS was using for other metric profile and caliper-transport events with LTI 1.1 and Caliper 1.0, the team outlined the specific negotiation around the event endpoint, and a security token was added to the existing Kaltura LTI launch, which was covered by a new permission in the Building Block manifest (Fig. 2).
Figure 3. Example of a Caliper MediaEvent from Kaltura.
Extending the Specification
To facilitate reporting from the LMS side, the team extended the MediaEvent profiles:
- Using object extensions to send the set of LMS identifiers for the user, course, and content item on launch. These needed to be sent back as is in the event payloads from Kaltura, to allow the reporting to “roll up” correctly in the LMS.
- Sending a periodic “still watching” event from Kaltura, so that the “drop off” rate can be measured, without having to try to trap every possible browser interaction or network issue that might cause a student to stop or be unable to watch.
Figure 4. Student Video Usage Data from Kaltura visualized in Blackboard Learn (SaaS) via Caliper integration.
Blackboard and Kaltura intend to bring emit capability to the enterprise deployments of Learn. Additionally, Kaltura intends to begin supporting Assessment/Assessment Item metric profile for new features of the video platform.
This educational analytics strategy will provide benefit to teachers, institutional researchers, and heads of programs using richly designed online curricula and tools that provide access to more student activity information in a familiar interface, regardless of the tool that provides the learning material. As these kinds of integrations become more widespread, Caliper-based solutions like this have the potential to alleviate the existing data silo problem and give all those supporting student learning a much more complete picture of student activity, with less friction and difficulty in mining the data.
For More Information
All IMS Global standards are free to download and free to use. To claim conformance to an IMS standard, an implementing organization must complete the conformance certification process (which requires IMS Global membership).
To find out how to use IMS specification documents, please click on the following link: https://www.imsglobal.org/usingimsdocuments.html
All products that have achieved IMS conformance certification of any kind are listed and updated here: https://www.imsglobal.org/cc/statuschart.cfm