On Monday, August 15, at the IMS Global quarterly meeting at Utah Valley University, the leaders of the Advanced Distributed Learning (ADL) initiative, responsible for the Experience API (xAPI), came together with the leaders of the Caliper Learning Analytics Framework (Caliper) from IMS Global. ADL has joined as a Contributing Member of IMS Global and the parties are committed to exploring a unified path for xAPI and Caliper. This may mean alignment at some level or potentially even convergence.
There were roughly 50 experts in the room for an entire day on August 15, and I believe it is fair to say that both ADL and IMS were very pleased with the eagerness of participants to cooperate and depth of discussion. Presentations from the meeting are posted here and there should be a synopsis of the comparison posted soon.
At the next IMS quarterly meeting November 7-10, 2016 at Arizona State University, there will be a follow-up session on Wednesday November 9 open to the public (registration required). I expect it to be another great session that should lead to some tangible prioritization and next steps. On November 8 there will also be a day-long Summit on Creating an Educational Analytics Ecosystem.
IMS and ADL are committed to collecting and achieving public input as the process moves forward. We are likely to create a public forum to encourage the public community to comment and also help crosswalk the information models. Stay tuned to ADL and IMS announcements and twitter feeds (IMS twitter feed is @LearningImpact). Currently there is a survey for those that have views on the potential convergence of the specifications here if you would like to participate.
I attended the entire August 15th session and would like to provide my sense of the big picture. I’m not the technical expert, but I have a long history in the applications of standards, including those that evolved into SCORM, and of course the dramatic growth of IMS standards adoption the last 10 years in the education sector.
I am extremely impressed with both the xAPI and Caliper work. While they are perceived by many as “solving the same problem” my own take is that this is largely because there is still quite a lot of variability in the Learning Analytics field in general. Thus, almost any set of technical work that deals with transcribing, sending, logging and analyzing data during any form of e-learning is considered “learning analytics.” From my perspective, while xAPI and Caliper certainly have some overlap, they are largely complementary at this phase in their respective evolution.
At the risk of over-simplifying things, my impressions of the strengths of each specification are captured in the table above (while there are four “strengths” shown for each specification they are not meant to be compared in four dimensions - the above are just lists of the primary strengths). The strength of xAPI is as a way for a single application to log freeform natural language statements and for that data store to be retrievable by the logging entity. The strength of Caliper is as a way to aggregate agreed upon events across a set of applications to enable processing across the aggregation of data.
It is also very clear that while neither specification “requires” the body of work from which they came (SCORM for xAPI and IMS standards, like LTI and OneRoster for Caliper) that these foundations greatly influence each. For instance, in the case of Caliper the co-existence of the other standards in an implementation means that Caliper can simply complement a slew of outcomes and context data that are already flowing via LTI. Again, because the Learning Analytics field is rather loosely defined at this point with widely varying “use cases” one can easily get confused about whether they even need Caliper or should just be using LTI to it’s fullest potential (for instance, LTI can already send data to an endpoint that is not the LMS).
I will also note that in the IMS Communities of Practice relating to data and analytics in IMS, we are seeing a very high priority need to simply be able to see all the data visualizations in one place. The level of data processing may be fairly minimal – but the humans in the equation need a better way to integrate visualizations that may already be available from various products they are using.
Bottomline, in my humble opinion, is that as we move forward on alignment and/or convergence of xAPI and Caliper I think we need to consider the use cases more carefully and, in essence, better define the “categories” or “scenarios” of Learning Analytics. While that may seem like “a step backwards” well, I think we need more clarity. The good news is that I don’t believe we will have any problem given the wealth of actual implementations we are working with in IMS and ADL.
Our goal in IMS is to enable serious school districts, higher ed institutions, states and nations to implement high impact learning analytics use cases and to enable a plug and play ecosystem of products that provide the data needed to solve those use cases. Either specification is good enough to begin doing some work. But, in order to do what we need to do we need a very large interoperable ecosystem of products, users, and researchers that can all invest together to get to the understanding we wish to collectively achieve. I would see elements of xAPI and elements of Caliper as both being essential in that goal.