Learning Impact Blog

Community leadership for more effective use of technology in service to education

This blog post is an interview with IMS’s Rob Abel to get to the bottom of IMS’s recent announcement of its Caliper Analytics Framework.

See related post on "small data".

Feel free to post additional questions and Rob will answer them (we hope)!

Q1: Is this project/announcement a big deal?

A1: We’ve got a lot of very impactful stuff going on in IMS these days, but enabling widespread adoption of analytics is one of the top priorities of IMS – with a mandate coming right from the IMS Board of Directors. But, perhaps more importantly, if we want to gain the full potential benefit of analytics and dashboards in education we need to make sure it is relatively easy to enable the transmission of data from any applications that can provide useful data. Interoperability standards, if done correctly, can help enable this.

Q2: There is lots of work going on in analytics, dashboards, etc.  Why is this a credible entry into the analytics space by IMS?

A2: Several reasons. IMS has had a relatively unique focus on one of the potentially more fruitful but challenging data collection areas of learning applications and platforms. IMS knows this turf well and brings a large critical mass of members that cover a wide range of product categories and institutional needs. IMS also has a large installed base of applications already using its core standards, such as LTI (Learning Tools Interoperability), Common Cartridge, LIS (Learning Information Services)  and QTI (Question and Test Interoperability). In other words, data is already flowing via these specifications, which are providing conduits upon which more data can ride.

Q3: What is the focus of the analytics initiative versus other IMS specification work?

A3: IMS has a bunch of fast moving task forces and leadership groups that work on applications of standards. This analytics work is coming from such a group that has been in existence about a year, but leveraging off of years of work by IMS in specifications for outcomes data for a variety of purposes – ranging from scores to gradebooks to assessment item data.  The purpose of the analytics effort is to actualize many implementations of analytics feeds in as many products as rapidly as possible, adding a few new bits and pieces, but largely leveraging existing work. In fact, the first proof of concept demonstrations will come very soon – at the next IMS quarterly meeting the week of November 4.  There will be a Summit day there, Thursday, November 7, that will also focus on analytics – both the institutional and supplier perspectives.

Q4: What is the relationship with IMS LTI (Learning Tools Interoperability)?

A4: There’s a short answer and a longer answer.  The short answer is that we expect this analytics work to finalize some work on outcomes (namely a very robust gradebook) that has been proposed for LTI & LIS for awhile but not officially released yet and that we expect the proliferation of LTI-enabled apps and learning platforms to be a natural starting point for data exchange. The longer term has two components. The first is that we are leveraging the work of IMS members in specific LTI app/product categories to help develop the Learning Metric Profiles referenced in the Caliper whitepaper. The second is that we will be introducing a variety of LTI Services that will enable data to go to many different destinations from many sources whether or not LTI was used as the launch mechanism for an app. So, for instance, enabling apps to send data to an analytics storage, dashboard or personal data vault whether or not it was launched by an LMS/learning platform.

Q5: Why is this project developing and releasing the Sensor API?

A5: APIs can make implementation a little easier – especially if a large number of suppliers use the same APIs. IMS is now releasing APIs as best practices with many of its specifications now. Please note that an “API” is by definition programming language specific and a good standard is not.  The standard is the underlying guts – that’s the hard part.

Q6: What if a product company has already developed an API for some category of data transmission – can that still be used with Caliper?

A6: Maybe. One of the cool things about the IMS specs and the development process behind them is that we can work with leading suppliers who already have services/APIs to see if we can “map them” on top of the IMS specifications. You may have developed some APIs that are now experiencing good market adoption for a specific type of service. IMS can potentially work with your organization to harmonize that with Caliper and the LTI services. Please contact us at:

Q7: What if IMS can’t get suppliers to agree on the Learning Metric Profiles?

A7: Well, we wouldn’t be doing this if we were not already seeing some excellent convergence. But, we also want to encourage and fully expect there to be extensions, both public and private, that IMS will capture in a registry.  Thus we can have the stuff that everyone agrees on and the stuff that is new, above and beyond that is either publically sharable or not. That’s how innovation in data and analytics is enabled by all of this.

Q8: What about applications that are kind of out of the learning domain, like CRM (Customer Relationship Management) systems?

A8: We see absolutely no reason why Caliper cannot add Metric Profiles for classes of systems like this and add into the mix. The Caliper Framework should be applicable to almost any type of system.

Q9: What if my analytics product wants to suck in every possible data and user interaction possible?

A9: Yes, big data. If you want to do that across more than one system you still need an agreed upon analytics feed. Caliper will cover that, even if a private solution is needed at first (see A7 above).

Q10: Will U.S. K-12 initiatives such as InBloom or Ed-Fi benefit from this work?

A10:  Yes, they certainly could! Caliper data can go to/from anywhere.