Why 1EdTech Conformance and Certification?

Hi Everyone,

Based on questions I get from time to time in my travels and emails I thought it would be good to post on why IMS is doing conformance and certification the way it is these days. This is important for both suppliers and end-users to understand.

A little bit of history. Prior to about 12 months ago, there was literally no way to be "conformant" to IMS specifications and standards. That's because there were no tests of any type on any IMS specifications. IMS tried to establish a conformance test in the 2004-6 timeframe and there were literally no takers. So, if you are a buyer - or were a buyer - and asked for IMS conformance in an RFP - and a supplier claimed they were IMS "compliant" - well, guess what - you were asking for something that did not exist and were getting promised something that could not be delivered. The best that you could have gotten was an implementation of IMS Content Packaging, QTI, or IMS Enterprise that was based on the specification, but left a lot of interpretation up to both products on each side of an "interoperable exchange" based on them.

SCORM, of course, includes a bunch of IMS work that is refined to the point of testability, and of course, there are SCORM conformance tests from independent testing centers. The tests have been widely criticized for not ensuring interoperability over the years - but they were definitely a step forward from the mayhem of many suppliers independently interpreting the IMS specs.

Fast forward to about 3 years ago, we here at IMS began discussing with our members what they wanted to see in terms of support for conformance. This was a really important discussion because by far the biggest criticism I heard about IMS when I came here in 2006 was - "well, we have all these specs but we have very little interoperability." The members made it very clear that what they wanted was technical support to make interoperability happen in practice. This did not so much mean a conformance marketing seal as real technical tools and support - stuff that costs money to do on their own and they would rather have a place to cooperate and "share the pain."

So, from about that time - 3 years ago or so - IMS has been putting in place what the members asked for. As of today, I am pleased to report that 21 certifications for either Common Cartridge or Basic LTI have been "granted" - and there are at least 10 more in the works - you can always check the current list here. We were very fortunate that about 9 of the members got together and through IMS invested in the development of a Common Cartridge test system. Microsoft invested a lot in an open source Common Cartridge player and a SCORM to Common Cartridge converter. In short, over the last 3 years the members and our staff have developed numerous support tests, tools, and code to help you implement Common Cartridge and Basic LTI.

However, it's important to understand that tools and tests are just part of ensuring interoperability. True to what our members asked for, our staff now provide hands-on support in helping do the testing and running to ground issues that arise. This means for current versions of these specs and the evolving versions (it is very clear that we will be on annual revision cycles for the foreseeable future as the core Digital Learning Services Standards take hold and fold in more aspects of IMS's broader portfolio of work ). This support includes hosted versions of the leading platforms so we can facilitate testing for the members on a bunch of platforms in one place.

This is the norm for IMS going forward. While we offer this only for Common Cartridge and Basic LTI today, it will be standard fare for most IMS releases going forward.

So, what's up with asking organizations to join an "Alliance" to get this support. Most of you reading this probably know that we have been emphatic about using the membership-based Common Cartridge/LTI Alliance as the vehicle by which to get support. And, we are doing the same for Learning Information Services (LIS). The reasons are simple and are twofold.

The first reason is that we are attempting to change the above described historical behavior in the market of suppliers running around claiming IMS conformance on their own accord when in fact interoperability was not ensured. Through the Alliance structure IMS not only has hands-on involvement with each supplier, but also can work within the membership to resolve any disputes that arise (like product x and product y both think they are conformant but it is not working at institution/district A), and keep implementing organizations involved and informed about evolution. Please note that whereas in the old IMS there might have been the rechartering of a new workgroup to get from Common Cartridge v1.0 to v1.1, that DOES NOT HAPPEN any longer. The Common Cartridge APMG are the voting members in IMS that are implementing Common Cartridge and they will be evolving Common Cartridge on an ongoing basis - and this is important - in conjunction with the Alliance members.

The second reason is that we are always trying to gauge market support for what we are doing in IMS. So, we charge a minimal $500 - $3000 a year annual dues for an Alliance. Quite honestly this is more of an intent of seriousness to engage than it is of any financial consequence to IMS - at least at this stage. But, if the success of the work scales, it will be important to have some financial support in place to sustain this. For those that are individual dabblers who can't afford any fees - well, there is lots of free stuff around the Internet on literally all the IMS stuff, and, we often have staff work with you as well if we think it makes sense. Just ask us.

So, this is a very different world than 5 years ago in terms of what it means to implement an IMS specification. First of all, IMS has significant "skin in the game" of actually achieving interoperability in practice. And, this goal really defines our work. The specifications are still important as a basis for this, but specification development is just one piece of delivering on our value proposition.

For end-users, you can now demand conformance to IMS specifications that have conformance marks with confidence. Today (August 15, 2010) that is Common Cartridge v1.0 and Basic LTI v1.0. IMS will stand behind those products with marks we issue.

And, if you are an end-user organization that wants help determining how to "ask for" and "make sure you get" compliant products, we encourage you to join IMS. If you do we can justify supporting your institution or district in a very hands on way - as we are doing now with the New York City Department of Education in the iLearnNYC, along with many other IMS institutional members. The cost of an IMS dues is much lower than what you will pay a consulting organization or "standards expert" to help you write your RFP. I have been both shocked and amazed at the organizations out there - some very well known - who will accept payment to advise on IMS specifications who have literally had zero interaction with IMS over the last 5 years - if ever. So, end-users beware of those claiming IMS expertise. Look for them on our membership list here to see which organizations really have IMS expertise.

And, for those suppliers who would prefer to go it alone and stay unaffiliated with IMS - well, I can only say "Good luck with that!" For a few dollars a year you can benefit from literally millions of dollars investment made by the IMS Contributing Members in what is available in the Alliances and deliver higher value to your customers in terms of interoperability that actually works and has a broad base of suppliers and end-users to stand behind it.

I'm pretty confident that there is not a better approach out there to get where we really need to be to establish the open architecture for learning technology innovation. But, we are always listening and encourage your thoughts and questions.

Many thanks to the IMS members - now nearing 160 - up from 50 not too long ago - many of whom are avid competitors in the market - but have come together in a very unique and inspiring way for this special cause.

- Rob Abel