Sharebar?

Digital Assessment in IMS Ecosystems

 

IMS Chief Architect Dr. Colin SmytheIMS TECH TALK

Contributed by Dr. Colin Smythe, IMS Chief Architect

 

Digital Assessment in IMS Ecosystems: The Set of IMS Specifications Making Online Assessment a Reality

In June 2000, IMS published version 1.0 of the Question and Test Interoperability® specification—or QTI® as we all know it. This specification described a structure for exchanging tests and questions to enable the authoring and distribution of online assessments. The following 36 months saw several new versions resulting in the publication of version 1.2.1 in March 2003. This version was later included in the IMS Common Cartridge® 1.0 specification. Our work from 2004 to 2015 resulted in QTI versions 2.0, 2.1, and 2.2. Between QTI 2.1 and 2.2, IMS released the Accessible Portable Item Protocol® (APIP®) 1.0, which added new accessibility features to the core QTI specification. In September 2015, the IMS Technical Advisory Board (TAB) authorized the Charter for QTI version 3.0 and, in July 2020, made available the Candidate Final Public documents for public review.

The longevity of the QTI specification is based on the early research undertaken by long-time IMS Contributing Member ETS and published as the Four Process Architecture. The four-process architecture gives rise to two of the key QTI structures: Assessment and Item. Notice that we use the term ‘Item’ instead of ‘Question.’ This is because an Item includes more than just the actual question: for example, the response processing rules for determining the correctness of the learner’s response (their answer); alternative content to reflect accessibility needs; and much more. An assessment, test, or quiz is the combination and organization of a set of Items. QTI also defines Section. A Section is a collection of Items and Sections. Sections enable the creation of flexible and adaptive assessments to reflect the assessment pedagogy being used and ease the management of the assessment structure. An Item is based upon one or more Interactions. QTI has nineteen predefined interactions to support the most typical question types, e.g., multiple-choice, hotspot, etc.

IMS was an early adopter of the use of XML for data exchange. Document Type Definitions were used for the 1.x versions of QTI to validate the QTI XML instances, but XML Schema Definitions (XSDs) are used for all later versions. Certification of compliant products is an essential aspect of the IMS specification support process. For XML-based specifications, IMS uses a proprietary online XML validator for certification. Unlimited access to this validator is available to all members, and many make extensive use of this benefit as part of their content development validation process. For QTI versions 2.1, 2.2, and 3.0, the specification is composed of several individual specifications:

  • QTI Assessment, Section & Item (ASI) – the core definition of the QTI structures and their representation in XML

  • QTI Metadata – the QTI-specific metadata to be included in the accompanying content packaging manifest file

  • QTI Results Reporting – the information used to represent the exchange of the results awarded to a learner as part of the response and outcomes processing as defined in the core ASI

  • QTI Usage Data & Item Statistics – the information used to describe the usage of the QTI Items in terms of a variety of standardized statistics

Several more IMS and non-IMS specifications are used to provide the complete QTI interoperability capability:

These specifications are combined at the level of the XML binding to create a single zip file as required in the QTI Profile of IMS Content Packaging.

Actionable e-Assessment is an important aspect of IMS Ecosystems. While QTI is the keystone specification, many IMS specifications can be combined to support an extensive range of assessment processes and workflows.

This broader set of specifications is:

As can be seen, IMS has a thriving specification development activity in e-Assessment. We will continue to improve how our specifications can be combined to support your assessment processes and workflows.