IMS Question and Test Interoperability (QTI): Assessment, Section and Item Information Model Version 2.2.4

IMS Final Release
Version Final 1.0

IPR and Distribution Notices

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the specification set forth in this document, and to provide supporting documentation.

IMS takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on IMS's procedures with respect to rights in IMS specifications can be found at the IMS Intellectual Property Rights web page: http://www.imsglobal.org/ipr/imsipr_policyFinal.pdf.

Copyright © 2022 IMS Global Learning Consortium. All Rights Reserved.

Use of this specification to develop products or services is governed by the license with IMS found on the IMS website: http://www.imsglobal.org/speclicense.html.

Permission is granted to all parties to use excerpts from this document as needed in producing requests for proposals.

The limited permissions granted above are perpetual and will not be revoked by IMS or its successors or assigns.

THIS SPECIFICATION IS BEING OFFERED WITHOUT ANY WARRANTY WHATSOEVER, AND IN PARTICULAR, ANY WARRANTY OF NONINFRINGEMENT IS EXPRESSLY DISCLAIMED. ANY USE OF THIS SPECIFICATION SHALL BE MADE ENTIRELY AT THE IMPLEMENTER'S OWN RISK, AND NEITHER THE CONSORTIUM, NOR ANY OF ITS MEMBERS OR SUBMITTERS, SHALL HAVE ANY LIABILITY WHATSOEVER TO ANY IMPLEMENTER OR THIRD PARTY FOR ANY DAMAGES OF ANY NATURE WHATSOEVER, DIRECTLY OR INDIRECTLY, ARISING FROM THE USE OF THIS SPECIFICATION.

Public contributions, comments and questions can be posted here: www.imsglobal.org/forums/ims-glc-public-forums-and-resources.

© 2022 IMS Global Learning Consortium, Inc.
All Rights Reserved.

Trademark information: http://www.imsglobal.org/copyright.html

Document Name: IMS Question and Test Interoperability (QTI): Assessment, Section and Item Information Model v2.2.4

Revision: 18th March, 2021

Abstract

This document is the core part the IMS Global Question and Test Interoperability (QTI) v2.2 specification. This document describes the information model for items, sections and whole tests used in assessment. Formally these are known as the Assessment, Section and Item (ASI) features.

The IMS QTI specification describes a data model for the representation of question (assessmentItem) and test (assessmentTest) data and their corresponding results reports. Therefore, the specification enables the exchange of this item, test and results data between authoring tools, item banks, test constructional tools, learning systems and assessment delivery systems. The data model is described abstractly, using the Unified Modeling Language (UML) to facilitate binding to a wide range of data-modelling tools and programming languages, however, for interchange between systems a binding is provided to the industry standard eXtensible Markup Language (XML) and use of this binding is strongly recommended. The IMS QTI specification has been designed to support both interoperability and innovation through the provision of well-defined extension points. These extension points can be used to wrap specialized or proprietary data in ways that allows it to be used alongside items that can be represented directly.

Table of Contents

Abstract

1. Introduction

2. Key Data Model Features

3. Root Attribute Descriptions

4. Root Class Descriptions

5. Data Class Descriptions

6. Abstract Class Descriptions

7. Derived Class Descriptions

8. Enumerated Vocabulary Descriptions

9. Miscellaneous Descriptions

10. Extending and Profiling the Data Model

References

Appendix A Modelling Concepts and Terms

About this Document

List of Contributors

Revision History

List of Figures

List of Tables

1. Introduction

This Section is NOT NORMATIVE.

1.1. Scope and Context

The IMS QTI Assessment, Section and Item Information is a part of the broader QTI specification. This document should be used in conjunction with the following documents:

• IMS QTI v2.2: HTML5 Information Model and XSD Binding v1.0 [QTI, 21b] - the details of the new HTML5 functionality added to QTI;
• IMS QTI v2.2: Metadata Information Model and XSD Binding v1.0 [QTI, 16a] - the details of the QTI Metadata functionality and XSD binding;
• IMS QTI v2.2: Results Reporting Information Model and XSD Binding v1.0 [QTI, 16b] - the details of the QTI Results Reporting functionality and XSD binding;
• IMS QTI v2.2: Usage Data Information Model and XSD Binding v1.0 [QTI, 16c] - the details of the QTI Usage Data functionality and XSD binding;
• IMS QTI v2.2: ASI XSD Binding v1.0 [QTI, 21a] - the details of the binding of QTI in XML;
• IMS QTI v2.2 Best Practice and Implementation Guide v1.0 [QTI, 15] - key information on how to adopt and implement the specification;
• IMS Portable Custom Interactions v1.0 [PCI, 14] - the best practices for the creation of new types of items using an interoperable format.

This QTI ASI v2.2 specification supersedes the QTI v2.1 versions. The QTI Information Model is bound to an XML Schema Definition (XSD). This document and the accompanying XSD is generated using the IMS Global Binding Auto-generation Tool-kit (I-BAT) [IBAT, 06]. A significant number of new features have been added in v2.2, namely:

• Shared Stimulus - this allows key reference content to be shared by more than one Item and also for that content workflow to be directly supported by the QTI interoperability;
• Bidirectional Text - enables 'left-to-right' and 'right-to-left' text presentation;
• Speech Synthesis Markup Language (SSML) - this allows text to have direct SSMLv1.1 annotation;
• WAI-ARIA on the HTML tags - this allows the HTML to support ARIA accessibility information;
• HTML4 tags - support for the HTML4 tags 'bdo', and 'label' (the latter is placed in the HTML5 namespace to avoid conflict with the other new 'label' element);
• HTML5 tags - support for the HTML5 tags 'article', 'aside', 'audio', 'bdi', 'figure', 'figcaption', 'footer', 'header', 'ruby', 'nav', 'section', 'source', 'track', and 'video';
• HTML5 extensibility - support for the 'data-*' attributes;
• MathML 3 - providing support for both MathMl 2.0 and MathMl 3.0;
• CSS 3 - providing support for both CSS2.1 and CSS 3.0.

There have also been several changes to established features, namely:

• GapImg Class - add support for the 'top' and 'left' characteristics;
• GapText Class - considerable extension of the permitted child tags to support formatted text;
• GraphicGapMatchInteraction Class - allow 'gapText' as well as 'gapImg' children;
• TextEntryInteraction Class - add support for the 'format' characteristic;
• MediaInteraction Class - add support for the 'coords' characteristic;
• InlineChoice Class - considerable extension of the permitted child tags;
• InlineChoiceInteraction Class - add support for the, new, 'label' element (not the same as the HTML4 element);
• OutcomeDeclaration Class - add support for the 'externalScored' characteristics;
• GapMatchInteraction and GraphicGapMatchInteraction - add support for the 'minAssociations' and 'maxAssociations' characteristics;
• Data-typing - many data-types for characteristics have been changed to enforce the intentions of the information model more accurately.

1.2. Conventions

All sections marked as non-normative, all authoring guidelines, diagrams (with the exception of the UML diagrams), examples, and notes in this specification are non-normative. Everything else in this specification is normative.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119]. This means that from the perspective of conformance:

• IMS considers nonconformant any implementation of this specification that fails to implement a MUST/REQUIRED/SHALL requirement or fails to abide by a MUST NOT/SHALL NOT prohibition;
• SHOULD/SHOULD NOT/RECOMMENDED statements constitute a best practice; ignoring a best practice does not violate conformance but implementers should carefully consdier a decision to disregard such guidance;
• MAY/OPTIONAL statements indicate that implementers are entirely free to choose whether or not to implement the option.

The Conformance and Certification Guide for this specification may introduce greater normative constraints than those defined here for specific service or implementation categories.

The SHOULD/SHOULD NOT/RECOMMENDED statements MUST NOT be used in any document, or section of a document, that is responsible for defining the information model and/or the associated bindings and/or conformance and certification.

1.3. Structure of this Document

The structure of the rest of this document is:

1.4. Nomenclature and Definitions

2. Key Data Model Features

2.1. The Structure of the Data Model

The QTI v2.2 specification has four components that provide the total data interoperability capability:

• QTI Assessment, Section and Item - the core that defines the actual tests and/or question data exchange (including the content itself);
• QTI Metadata - QTI-specific metadata data;
• QTI Usage Data - statistical data about the usage of the Assessment and/or Items;
• QTI Results Reporting - reporting of the results assigned by outcomes and response processing of the Assessment and/or Items.

2.1.1. Assessments, Sections and Tests

This is the core functionality of the QTI specification and is the component that is addressed specifically by this document. This specification defines the exchange of a single Assessment (also known as a Test and Quiz), Sections and/or Items. Structurally an Assessment consists of one or more TestParts; each TestPart consists of one or more Sections; each Section consists of one or more Sections (hence a Section is recursive) and/or Items. This specification also permits the seperate exchange of an Outcome Declaration, a Response Processing Template and content Stimulus. The data model root classes are:

• AssessmentTest - for a test;
• AssessmetSection - for a Section;
• AssessmentItem - for an Item;
• AssessmentStimulus - for content stimulus;
• OutcomeDeclaration - for an outcome declaration;
• ResponseProcessing - for a response processing template.

This data model consists of two XSD bindings that are linked using namespacing. See the QTI HTML5 documentation [QTI, 21b] for the definition of the HTML5-based content that is used with the QTI ASI definition. The data exchange is achieved as a zip package as defined in [CP, 12].

2.1.2. Metadata

This is the QTI-specific metadata. In QTIv2.2 the QTI-specific metadata [QTI, 16a] is aligned with the IEEE Learning Object Metadata (LOM) in accordance with the IMS Metadata Best Practice and Implementation Guide for [IMD, 06]. The IEEE LOM standard defines a set of metadata elements that can be used to describe learning resources, but does not describe assessment resources in sufficient detail. The application profile provided in this document therefore extends the IEEE LOM to meet the specific needs of QTI developers wishing to associate metadata with items (as defined by the accompanying Item Information Model). QTIv2.2 further extends this to enable the description of tests, pools and object banks.

This data model has its own XSD binding.

2.1.3. Usage Data

QTI Usage Data defines the exchange of statistical information about the usage of a set of items. The corresponding documentation [QTI, 16d] provides detailed information about the data model. While the expectation is that the original definition of the corresponding Test and Items is based upon the use of the QTIv2.2 Assessment, Section and Item specification, this is not a pre-requisite for the use of this usage data representation.

The root class is 'UsageData'. Usage data, most commonly item statistics, do not form part of an assessmentItem directly because they always relate to some context or domain in which the statistics are valid. Therefore, this specification defines a separate class for describing these statistics. Each statistic refers to both its context and to the assessmentItem(s) it relates to. Therefore, instances of this class are bound and packaged separately for interoperability.

This data model has its own XSD binding.

2.1.4. Results Reporting

QTI Result Reporting defines the exchange of reporting the results of an assessment. The corresponding documentation [QTI, 16b] provides detailed information about the model and specifies the associated requirements on delivery engines. While the expectation is that the original definition of the corresponding Test and Items is based upon the use of the QTIv2.2 Assessment, Section and Item specification, this is not a pre-requisite for the use of this result reporting representation.

The root class is 'AssessmentResult'. An AssessmentResult is used to report the results of a candidate's interaction with a test and/or one or more items attempted. Information about the test is optional, in some systems it may be possible to interact with items that are not organized into a test at all. For example, items that are organized with learning resources and presented individually in a formative context.

This data model has its own XSD binding.

2.2. Items

In this specification an assessment item encompasses the information that is presented to a candidate and information about how to score the item. Scoring takes place when candidate responses are transformed into outcomes by response processing rules. It is sometimes desirable to have several different items that appear the same to the candidate but which are scored differently. In this specification, these are distinct items by definition and must therefore have distinct identifiers. To help facilitate the exchange of items that share significant parts of their presentation this specification supports the inclusion of separately managed item fragments in the itemBody. The core functionality of an Item is defined using the classes:

• AssessmentItem;
• ItemBody.

2.2.1. The Item Session Lifecycle

An item session is the accumulation of all the attempts at a particular instance of an item made by a candidate. In some types of test, the same item may be presented to the candidate multiple times e.g. during 'drill and practice'. Each occurrence or instance of the item is associated with its own item session. Figure 2.1, illustrates the user-perceived states of the item session. Not all states will apply to every scenario, for example feedback may not be provided for an item or it may not be allowed in the context in which the item is being used. Similarly, the candidate may not be permitted to review their responses and/or examine a model solution. In practice, systems may support only a limited number of the indicated state transitions and/or support other state transitions not shown here.

2.2.2. Item Variables

Item variables are declared by variable declarations: response, outcome or template variables. All variables must be declared except for the built-in session variables referred to below which are declared implicitly and must not be declared. The purpose of the declaration is to associate an identifier with the variable and to identify the runtime type of the variable's value. An item variable may have no value at all, in which case it is said to have the special value NULL. For example, if the candidate has not yet had an opportunity to respond to an interaction then any associated response variable will have a NULL value. Empty containers and empty strings are always treated as NULL values. The declaration of variables is defined by the classes:

• ResponseDeclaration;
• OutcomeDeclaration;
• TemplateDeclaration.

2.2.2.1. Response Variable

Response variables are declared by response declarations and bound to interactions in the itemBody. Each response variable declared may be bound to one and only one interaction. At runtime, response variables are instantiated as part of an item session. Their values are always initialized to NULL (no value) regardless of whether or not a default value is given in the declaration. A response variable with a NULL value indicates that the candidate has not offered a response, either because they have not attempted the item at all or because they have attempted it and chosen not to provide a response. If a default value has been provided for a response variable then the variable is set to this value at the start of the first attempt. If the candidate never attempts the item, in other words, the item session passes straight from the initial state to the closed state without going through the interacting state, then the response variable remains NULL and the default value is never used.

Implementors of Delivery Engine's should take care when implementing user interfaces for items with default response variable values. If the associated interaction is left in the default state (i.e., representing the default value) then it is important that the system is confident that the candidate intended to submit this value and has not simply failed to notice that a default has been provided. This is especially true if the candidate's attempt ended due to some external event, such as running out of time. The techniques required to distinguish between these cases are an issue for user interface design and are therefore out of scope for this specification.

There are two built-in response variables: 'numAttempts' and 'duration'. These are declared implicitly and must not appear in a 'responseDeclaration'.

All Delivery Engines must maintain the value of numAttempts. This is a single integer that records the number of attempts at each item the candidate has taken. The value defaults to 0 initially and then increases by 1 at the start of each attempt.

Systems that support Time Dependent Items must also maintain the value of duration. The duration is defined as being a single float that records the accumulated time (in seconds) of all Candidate Sessions for all Attempts. In other words the time between the beginning and the end of the item session minus any time the session was in the suspended state. The resolution of the duration must be at least 1s and should be 0.1s or smaller. If the resolution is denoted by epsilon then each value of duration represents the range of values 'duration' <= 't' <= 'duration+epsilon'. In other words, duration values are truncated.

The value of duration for items that are not time dependent must not be used in any item or test-level expression, though systems should still report it in itemResults when it is known.

2.2.2.2. Outcomes Variable

Outcome variables are declared by outcome declarations. Their value is set either from a default given in the declaration itself or by a responseRule during responseProcessing.

Items that declare a numeric outcome variable representing the candidate's overall performance on the item should use the outcome name 'SCORE' for the variable. SCORE needs to be a float.

Items that declare a maximum score (in multiple response choice interactions, for example) should do so by declaring the 'MAXSCORE' variable. MAXSCORE needs to be a float.

Items or tests that want to make the fact that the candidate scored above a predefined treshold available as a variable should use the 'PASSED' variable. PASSED needs to be a boolean.

At runtime, outcome variables are instantiated as part of an item session. Their values may be initialized with a default value and/or set during responseProcessing. If no default value is given in the declaration then the outcome variable is initialized to NULL unless the outcome is of a numeric type (integer or float) in which case it is initialized to 0.

For Non-adaptive Items, the values of the outcome variables are reset to their default values prior to each invocation of responseProcessing. For Adaptive Items the outcome variables retain the values that were assigned to them during the previous invocation of response processing. For more information, see Response Processing (Subsection 2.5).

There is one built-in outcome variable, 'completionStatus', that is declared implicitly and must not appear in an outcomeDeclaration. Delivery Engines must maintain the value of the built-in outcome variable completionStatus, a single identifier. It starts with the reserved value "not_attempted". At the start of the first attempt it changes to the reserved value "unknown". It remains with this value for the duration of the item session unless set to a different value by a setOutcomeValue rule in responseProcessing. There are four permitted values:

• 'completed' - the learner has experienced enough of the item to consider it completed;
• 'incomplete' - the learner has not experienced enough of the item to consider it completed;
• 'not_attempted' - the learner is considered to have not used the item in any significant way;
• 'unknown' - no assertion on the state of completion can be made.

Any one of these values may be set during response processing. If an Adaptive Item sets completionStatus to completed then the session must be placed into the closed state, however an item session is not required to wait for the completed signal before terminating, it may terminate in response to a direct request from the candidate, through running out of time or through some other exceptional circumstance. Adaptive Items must maintain a suitable value and should set completionStatus to "completed" to indicate when the cycle of interaction, response processing and feedback must stop. Non-adaptive Items are not required to set a value for completionStatus, but they may do so. Delivery Engines are encouraged to use the value of completionStatus when communicating externally

2.3. The Content Model

Most of the content within QTI is used in the content of the AssessmentItem. The item body contains the text, graphics, media objects and interactions that describe the item's content and information about how it is structured. The body is presented by combining it with stylesheet information, either explicitly or implicitly using the default style rules of the delivery or authoring system.

The body must be presented to the candidate when the associated item session is in the interacting state. In this state, the candidate must be able to interact with each of the visible interactions and therefore set or update the values of the associated response variables. The body may be presented to the candidate when the item session is in the closed or review state. In these states, although the candidate's responses should be visible, the interactions must be disabled so as to prevent the candidate from setting or updating the values of the associated response variables. Finally, the body may be presented to the candidate in the solution state, in which case the correct values of the response variables must be visible and the associated interactions disabled.

The content model employed by this specification uses many concepts taken directly from [XHTML, 10]. In effect, this part of the specification defines a profile of XHTML. Only some of the elements defined in XHTML are allowable in an assessmentItem and of those that are, some have additional constraints placed on their attributes. Only those elements from XHTML that are explicitly defined within this specification can be used (see Subsection 2.3.2 XHTML Elements [XHTML, 10] for details). Finally, this specification defines some new elements which are used to represent the interactions and to control the display of Integrated Feedback and content restricted to one or more of the defined content views.

2.3.1. Basic Classes

Underpinning the content model are a number of abstract classes that are used to group elements of the body into categories that define peer-groups. These abstract classes are:

• BlockContentModel - the HTML block elements used in the context of the abstract classeses BlockGroup and BlockStaticGroup;
• BlockGroup - the broader set of Block content that adds the range of QTI interactions, plus other markup, to the base HTML Block content;
• BlockStaticGroup - a subset of BlockGroup that is used for the GapMatch and HotText interactions;
• FeedandTempBlockStatic - content for Template Blocks based upon static content;
• FeedbackBlockStatic - enables the FeedbackBlock class to include any sequence of content based upon the blockGroup, inline content, inline templates, inline feedback and printedVariables;
• FeedbackFlowStaticGroup - to enable the content to be defined for Modal and Test feedback;
• FeedbackInlineGroup - the content for the inline feedback;
• FlowContentModel - HTML flow content (not including QTI interactions). These are tags that can appear inside list items, table cells, etc. which includes block-type and inline-type elements;
• FlowGroup - HTML flow content including the set of QTI interactions. These tags that can appear inside list items, table cells, etc. which includes block-type and inline-type elements;
• FlowStaticGroup - HTML flow content that includes support for feedback and template-base content;
• InlineChoiceGroup - enables a simple run of text to be displayed to the user, may be subject to variable value substitution with printedVariable and may include feedback and template-based content;
• InlineContentModel - the set of HTML inline content tags to be used to create the corresponding HTML content;
• InlineGroup - elements that behave as spans of text, such as the contents of paragraphs and includes the inline QTI interactions;
• InlineStaticGroup - the set of HTML inline content tags plus a limited set of QTI-specific tags to be used to create the corresponding HTML content;
• ItemBodySelect - permits the mixing of 'rubricBock' with BlockContent within the ItemBody;
• ObjectFlowGroup - elements that can appear within an 'object';
• PromptStaticGroup - the content that is permitted within the 'prompt' element
• RubricBlockContentModel - defines the content that is permitted at the top-level wthin a 'rubricBlock' element;
• RubricTemplateBlockContentModel - defines the content that is permitted wthin a 'rubricBlock' element as a child of 'templateBlock';
• RubricTemplateInlineStaticGroup - defines the content that is permitted wthin a 'rubricBlock' element as a child of 'templateInline'.

2.3.2. XHTML Elements

The structural elements of the content model are taken from [XHTML, 10]. Only those characteristics listed here may be used (including attributes inherited from parent classes). By default, elements and attributes have the same interpretation and restrictions as the corresponding elements and attributes in [XHTML, 10].

2.3.2.1. Text Elements

The supported text elements are:

• 'abbr', acronym' and 'address';
• 'blockquote' and 'br';
• 'cite' and 'code';
• 'dfn' and 'div';
• 'em';
• 'h1', 'h2', 'h3', 'h4', 'h5' and 'h6';
• 'kbd';
• 'p' and 'pre';
• 'q';
• 'samp', 'span' and 'strong';
• 'var'.

2.3.2.2. List Elements

The supported list elements are:

• 'dl', 'dt' and 'dd';
• 'li';
• 'ol';
• 'ul'.

2.3.2.3. Object Elements

The supported object elements are:

• 'object'.

2.3.2.4. Presentation Elements

The supported presentation elements are:

• 'b' and 'big';
• 'hr';
• 'i';
• 'small', 'sub' and 'sup';
• 'tt'.

2.3.2.5. Table Elements

The supported table elements are:

• 'caption', 'col' and 'colgroup';
• 'table', 'tbody', 'td', 'tfoot', 'thead' and 'tr'.

2.3.2.6. Image Elements

The supported image elements are:

• 'img'.

2.3.2.7. Hypertext Element

The supported hypertext elements are:

• 'a'.

2.3.2.8. HTML5 Elements

The supported HTML5 elements are:

• 'article', 'aside' and 'audio';
• 'bdi';
• 'figcaption', 'figure' and 'footer';
• 'header';
• 'label';
• 'nav';
• 'rb', 'rp', 'rt', 'rtc' and 'ruby';
• 'section' and 'source';
• 'track';
• 'video'.

2.3.3. MathML

MathML defines a Markup Language for describing mathematical notation using XML [MathML2, 03], [MathML3, 14]. The primary purpose of MathML is to provide a language for embedding mathematical expressions into other documents, in particular into HTML documents.

It is often desirable to vary elements of a mathematical expression when creating item templates. Although it is impossible to embed objects such as printedVariable defined for that purpose within a math object the techniques described in this section can be used to achieve a similar effect. In MathML, numbers are represented either by the 'mn' or 'cn' tags, for presentation or content representation respectively. Similarly, 'mi' and 'ci' represent identifiers. If mathVariable is set in a template variable's declaration then all instances of 'mi' and 'ci' that match the name of the template variable must be replaced by 'mn' and 'cn' respectively with the template variable's value as their content. It is possible that this technique of expanding template variables will be extended to other elements of MathML in future. The set of classes that support the MathML are:

• MathML 2 (imported);
• MathML 3 (imported).

2.3.4. Variable Content

This specification defines two methods by which the content of an assessmentItem can vary depending on the state of the item session. The first method is based on the value of an outcome variable and the second uses templates. When using outcome variables the classes that support this method are:

• FeedbackBlock;
• FeedbackInline;
• RubricBlock;
• PrintedVariable.

The syntax of the format attribute is based on the format conversion specifiers defined in the C programming language [ISO 9899] for use with printf and related functions. Each conversion specifier starts with a '%' character and is followed by zero or more flag characters (#, 0, -, " " [space] and +), an optional digit string indicating the minimum field width, an optional precision (consisting of a "." followed by zero or more digits) and finally one of the conversion type codes: E, e, f, G, g, r, R, i, o, X, or x. These are interpreted according to the C standard with the exception of i, which may be used to format numbers in bases other than 10 using the base attribute, and r/R which round to the number of significant figures given by the precision in the same way as g/G except that scientific format is only used if the requested number of significant figures is less than the number of digits to the left of the decimal point.

2.3.5. Formatting Items with Stylesheets

QTI recommends the use of Cascading Style Sheets (CSS) for controlling the content formating. QTI supports CSS 2.1 [CSS2, 11] and CSS 3.0 [CSS3, 11]. The classes that define the associated stylesheet assignment are:

• Stylesheet.

Stylesheets are applicable to the content in the following classes:

• AssessmentItem;
• AssessmentStimulus;
• AssessmentTests;
• FeedbackBlock;
• ModalFeedback;
• RubricBlock;
• RubricBlockTemplateBlock;
• TemplateBlock;
• TemplateBlockFeedbackBlock;
• TestFeedback.

2.3.6. Accessibility

There are three accessibility features that are supported by QTI 2.2:

• Accessible Portable Item Protocol (APIP) - enables the APIP markup to be used;
• Speech Synthesis Markup Language (SSML) - enables text to have SSML annotation;
• WAI-ARIA - allows the ARIA annotations to be used on the HTML and interaction tags.

See Subsection 2.14 for more information on how Accessibility is supported.

2.3.7. Internationalization

As part of the QTIv2.2 new features, support for internationalization was added. The internationalization features now supported are:

• Bidirectional Text - provision of right-to-left and left-to-right text presentation;
• Ruby - these annotations are short runs of text presented alongside base text, primarily used in East Asian typography as a guide for pronunciation or to include other annotations;
• Vertical Writing - this is achieved by using CSS as recommended in the W3C's 'CSS Writing Modes Module Level 3' [W3C, 14] (at the time of QTI publication this W3C document was a Working Draft).

See Subsection 2.15 for more information on how Internationalization is supported.

2.4. Interactions

2.4.1. Simple Interactions

The simple interactions are:

• Choice Interaction;
• Order Interaction;
• Associate Interaction;
• Match Interaction;
• Gap Match Interaction.

2.4.1.1. Choice Interaction

The choice interaction presents a set of choices to the candidate. The candidate's task is to select one or more of the choices, up to a maximum of maxChoices. The interaction is always initialized with no choices selected. The choiceInteraction must be bound to a response variable with a baseType of identifier and single or multiple cardinality. The relevant classes are:

• ChoiceInteraction.

2.4.1.2. Order Interaction

In an order interaction the candidate's task is to reorder the choices, the order in which the choices are displayed initially is significant. By default the candidate's task is to order all of the choices but a subset of the choices can be requested using the maxChoices and minChoices attributes. When specified the candidate must select a subset of the choices and impose an ordering on them. If a default value is specified for the response variable associated with an order interaction then its value should be used to override the order of the choices specified here. By its nature, an order interaction may be difficult to render in an unanswered state, especially in the default case where all choices are to be ordered. Implementors should be aware of the issues concerning the use of default values described in the section on Response Variables. The orderInteraction must be bound to a response variable with a baseType of identifier and ordered cardinality only. The relevant classes are:

• OrderInteraction;
• SimpleChoice.

2.4.1.3. Associate Interaction

An associate interaction is a blockInteraction that presents candidates with a number of choices and allows them to create associations between them. The associateInteraction must be bound to a response variable with base-type pair and either single or multiple cardinality. The relevant classes are:

• AssociateInteraction;
• SimpleAssociableChoice.

2.4.1.4. Match Interaction

A match interaction is a blockInteraction that presents candidates with two sets of choices and allows them to create associates between pairs of choices in the two sets, but not between pairs of choices in the same set. Further restrictions can still be placed on the allowable associations using the matchMax attribute of the choices. The matchInteraction must be bound to a response variable with base-type directedPair and either single or multiple cardinality. The relevant classes are:

• MatchInteraction;
• SimpleMatchSet;
• SimpleAssociableChoice.

2.4.1.5. Gap Match Interaction

A gap match interaction is a blockInteraction that contains a number gaps that the candidate can fill from an associated set of choices. The candidate must be able to review the content with the gaps filled in context, as indicated by their choices. The gapMatchInteraction must be bound to a response variable with base-type directedPair and either single or multiple cardinality, depending on the number of gaps. The choices represent the source of the pairing and gaps the targets. Each gap can have at most one choice associated with it. The maximum occurrence of the choices is controlled by the matchMax attribute of gapChoice. The relevant classes are:

• GapMatchInteraction;
• Gap;
• GapChoice;
• GapText;
• GapImg.

2.4.2. Text-based Interactions

The text-based interactions are:

• InlineChoiceInteraction;
• TextEntryInteraction;
• ExtendedTextInteraction;
• HotTextInteraction.

2.4.2.1. Inline Choice Interaction

An inline choice is an inlineInteraction that presents the user with a set of choices, each of which is a simple piece of text. The candidate's task is to select one of the choices. Unlike the choiceInteraction, the delivery engine must allow the candidate to review their choice within the context of the surrounding text. The inlineChoiceInteraction must be bound to a response variable with a baseType of identifier and single cardinality only. The relevant classes are:

• InlineChoiceInteraction;
• InlineChoice.

2.4.2.2. Text Interaction

A textEntry interaction is an inlineInteraction that obtains a simple piece of text from the candidate. Like inlineChoiceInteraction, the delivery engine must allow the candidate to review their choice within the context of the surrounding text. The textEntryInteraction must be bound to a response variable with either single or record cardinality only. For single cardinality the baseType must be one of string, integer or float. For record cardinality the fields are 'stringValue', 'floatValue', etc. The relevant class is:

• TextEntryInteraction.

TextEntryInteraction is a String interaction and can be bound to numeric response variables, instead of strings, if desired. If detailed information about a numeric response is required then the string interaction can be bound to a response variable with record cardinality. The resulting value contains the following fields:

• stringValue - the string, as typed by the candidate;
• floatValue - the numeric value of the string typed by the candidate, as a float;
• integerValue - the numeric value of the string typed by the candidate if no fractional digits or exponent were specified, otherwise NULL. An integer;
• leftDigits - the number of digits to the left of the point. An integer;
• rightDigits - rightDigits: the number of digits to the right of the point. An integer;
• ndp- the number of fractional digits specified by the candidate. If no exponent was given this is the same as rightDigits. An integer;
• nsf - the number of significant digits specified by the candidate. An integer;
• exponent - the integer exponent given by the candidate or NULL if none was specified.

2.4.2.3. Extended Text Interaction

An extended text interaction is a blockInteraction that allows the candidate to enter an extended amount of text. The extendedTextInteraction must be bound to a response variable of single, multipe, ordered or record cardinality. If the response variable has record cardinality the fields in the records are 'stringValue' 'floatValue', etc. Otherwise it must have a baseType of string, integer or float. When bound to response variable with single cardinality a single string of text is required from the candidate. When bound to a response variable with multiple or ordered cardinality several separate text strings may be required, see maxStrings below. The relevant class is:

• ExtendedTextInteraction.

ExtendedTextInteraction is a String interactions and can be bound to numeric response variables, instead of strings, if desired. If detailed information about a numeric response is required then the string interaction can be bound to a response variable with record cardinality. The resulting value contains the following fields:

• stringValue - the string, as typed by the candidate;
• floatValue - the numeric value of the string typed by the candidate, as a float;
• integerValue - the numeric value of the string typed by the candidate if no fractional digits or exponent were specified, otherwise NULL. An integer;
• leftDigits - the number of digits to the left of the point. An integer;
• rightDigits - rightDigits: the number of digits to the right of the point. An integer;
• ndp- the number of fractional digits specified by the candidate. If no exponent was given this is the same as rightDigits. An integer;
• nsf - the number of significant digits specified by the candidate. An integer;
• exponent - the integer exponent given by the candidate or NULL if none was specified.

2.4.2.4. Hot Text Interaction

The hottext interaction presents a set of choices to the candidate represented as selectable runs of text embedded within a surrounding context, such as a simple passage of text. Like choiceInteraction, the candidate's task is to select one or more of the choices, up to a maximum of maxChoices. The interaction is initialized from the defaultValue of the associated response variable, a NULL value indicating that no choices are selected (the usual case). The hottextInteraction must be bound to a response variable with a baseType of identifier and single or multiple cardinality. The relevant classes are:

• HotTextInteraction;
• HotText.

2.4.3. Graphical Interactions

The graphical interactions are:

• HotspotInteraction;
• GraphicOrderInteraction;
• GraphicAssociateInteraction;
• GraphicGapMatchInteraction;
• SelectPointInteraction;
• PositionObjectInteraction.

2.4.3.1. Hotspot Interaction

A hotspot interaction is a graphical interaction with a corresponding set of choices that are defined as areas of the graphic image. The candidate's task is to select one or more of the areas (hotspots). The hotspot interaction should only be used when the spacial relationship of the choices with respect to each other (as represented by the graphic image) is important to the needs of the item. Otherwise, choiceInteraction should be used instead with separate material for each option. The delivery engine must clearly indicate the selected area(s) of the image and may also indicate the unselected areas as well. Interactions with hidden hotspots are achieved with the selectPointInteraction. The hotspot interaction must be bound to a response variable with a baseType of identifier and single or multiple cardinality.The relevant classes are:

• HotspotInteraction;
• HotspotChoice.

2.4.3.2. Graphic Order Interaction

A graphic order interaction is a graphic interaction with a corresponding set of choices that are defined as areas of the graphic image. The candidate's task is to impose an ordering on the areas (hotspots). The order hotspot interaction should only be used when the spacial relationship of the choices with respect to each other (as represented by the graphic image) is important to the needs of the item. Otherwise, orderInteraction should be used instead with separate material for each option. The delivery engine must clearly indicate all defined area(s) of the image. The order hotspot interaction must be bound to a response variable with a baseType of identifier and ordered cardinality.The relevant classes are:

• GraphicOrderInteraction;
• HotspotChoice.

2.4.3.3. Graphic Associate Interaction

A graphic associate interaction is a graphic interaction with a corresponding set of choices that are defined as areas of the graphic image. The candidate's task is to associate the areas (hotspots) with each other. The graphic associate interaction should only be used when the graphical relationship of the choices with respect to each other (as represented by the graphic image) is important to the needs of the item. Otherwise, associateInteraction should be used instead with separate Material for each option. The delivery engine must clearly indicate all defined area(s) of the image. The graphicAssociateInteraction must be bound to a response variable with base-type pair and either single or multiple cardinality. The relevant classes are:

• GraphicAssociateInteraction;
• AssociableHotspot.

2.4.3.4. Graphic Gap Match Interaction

A graphic gap-match interaction is a graphical interaction with a set of gaps that are defined as areas (hotspots) of the graphic image and an additional set of gap choices that are defined outside the image. The candidate must associate the gap choices with the gaps in the image and be able to review the image with the gaps filled in context, as indicated by their choices. Care should be taken when designing these interactions to ensure that the gaps in the image are a suitable size to receive the required gap choices. It must be clear to the candidate which hotspot each choice has been associated with. When associated, choices must appear wholly inside the gaps if at all possible and, where overlaps are required, should not hide each other completely. If the candidate indicates the association by positioning the choice over the gap (e.g. drag and drop) the system should 'snap' it to the nearest position that satisfies these requirements. The graphicGapMatchInteraction must be bound to a response variable with base-type directedPair and multiple cardinality. The choices represent the source of the pairing and the gaps in the image (the hotspots) the targets. Unlike the simple gapMatchInteraction, each gap can have several choices associated with it if desired, furthermore, the same choice may be associated with an associableHotspot multiple times, in which case the corresponding directed pair appears multiple times in the value of the response variable. The relevant classes are:

• GraphicGapMatchInteraction;
• GapImg;
• AssociableHotspot.

2.4.3.5. Select Point Interaction

Like hotspotInteraction, a select point interaction is a graphic interaction. The candidate's task is to select one or more points. The associated response may have an areaMapping that scores the response on the basis of comparing it against predefined areas but the delivery engine must not indicate these areas of the image. Only the actual point(s) selected by the candidate shall be indicated. The select point interaction must be bound to a response variable with a baseType of point and single or multiple cardinality. The relevant class is:

• SelectPointInteraction.

2.4.3.6. Position Object Interaction

The position object interaction consists of a single image which must be positioned on another graphic image (the stage) by the candidate. Like selectPointInteraction, the associated response may have an areaMapping that scores the response on the basis of comparing it against predefined areas but the delivery engine must not indicate these areas of the stage. Only the actual position(s) selected by the candidate shall be indicated. The position object interaction must be bound to a response variable with a baseType of point and single or multiple cardinality. The point records the coordinates, with respect to the stage, of the centre point of the image being positioned. The relevant classes are:

• PositionObjectInteraction;
• PositionObjectStage.

2.4.4. Miscellaneous Interactions

The set of miscellaneous interactions are:

• SliderInteraction;
• MediaInteraction;
• DrawingInteraction;
• UploadInteraction;
• CustomInteraction.

2.4.4.1. Slider Interaction

The slider interaction presents the candidate with a control for selecting a numerical value between a lower and upper bound. It must be bound to a response variable with single cardinality with a base-type of either integer or float. The relevant class is:

• SliderInteraction.

2.4.4.2. Media Interaction

The media interaction allows more control over the way the candidate interacts with a time-based media object and allows the number of times the media object was experienced to be reported in the value of the associated response variable, which must be of base-type integer and single cardinality. The relevant class is:

• MediaInteraction.

2.4.4.3. Drawing Interaction

The drawing interaction allows the candidate to use a common set of drawing tools to modify a given graphical image (the canvas). It must be bound to a response variable with base-type file and single cardinality. The result is a file in the same format as the original image. The relevant class is:

• DrawingInteraction.

2.4.4.4. Upload Interaction

The upload interaction allows the candidate to upload a pre-prepared file representing their response. It must be bound to a response variable with base-type file and single cardinality. The relevant class is:

• UploadInteraction.

2.4.4.5. Custom Interaction

Custom Interaction - the custom interaction provides an opportunity for extensibility of this specification to include support for interactions not currently documented. The relevant class is:

• CustomInteraction.

2.4.5. Alternative Ways to End an Attempt

The end attempt interaction is a special type of interaction which allows item authors to provide the candidate with control over the way in which the candidate terminates an attempt. The candidate can use the interaction to terminate the attempt (triggering response processing) immediately, typically to request a hint. It must be bound to a response variable with base-type boolean and single cardinality.

If the candidate invokes response processing using an endAttemptInteraction then the associated response variable is set to true. If response processing is invoked in any other way, either through a different endAttemptInteraction or through the default method for the delivery engine, then the associated response variable is set to false. The default value of the response variable is always ignored. The relevant class is:

• EndAttemptInteraction.

2.4.6. Alternative Ways to Provide Hints and Other Supplementary Material

The 'infoControl' element is a means to provide the candidate with extra information about the item when s/he chooses to trigger the control. The extra information can be a hint, but could also be additional tools such as a ruler or a (JavaScript) calculator. Unlike endAttemptInteraction, triggering infoControl has no consequence for response processing. That means that its triggering won't be recorded, nor the candidate penalised for triggering it. The relevant class is:

• InfoControl.

2.5. Response Processing

Response processing is the process by which the Delivery Engine assigns outcomes based on the candidate's responses. The state representation for the response processing is shown in Figure 2.2. The outcomes may be used to provide feedback to the candidate. Feedback is either provided immediately following the end of the candidate's attempt or it is provided at some later time, perhaps as part of a summary report on the item session.

2.5.1. Response Processing Templates

Response processing involves the application of a set of responseRules, including the testing of responseConditions and the evaluation of expressions involving the item variables. For delivery engines that are only designed to support very simple use cases the implementation of a system for carrying out this evaluation, conditional testing and processing may pose a barrier to the adoption of the specification.

To alleviate this problem, the implementation of generalized response processing is an optional feature. Engines that do not support it can instead implement a smaller number of standard response processors called response processing templates described below. These templates are described using the processing language defined in this specification and are distributed (in XML form) along with it. Delivery engines that support generalized response processing do not need to implement special mechanisms to support them as a template file can be parsed directly while processing the assessmentItem that refers to it.

Delivery engines that do not support generalized response processing but do support response processing mechanisms that go beyond the standard templates described below should, where possible, define templates of their own. Authors wishing to write items for those delivery engines can then refer to these custom templates. Publishing these custom templates will then ensure that these items can be used with delivery engines that do support generalized response processing.

There are three standard response processing templates available:

• MATCH CORRECT - the match correct response processing template uses the match operator to match the value of a response variable RESPONSE with its correct value. It sets the outcome variable SCORE to either 0 or 1 depending on the outcome of the test. A response variable with called RESPONSE must have been declared and have an associated correct value. Similarly, the outcome variable SCORE must also have been declared. The template applies to responses of any baseType and cardinality though bear in mind the limitations of matching more complex data types. This template shouldn't be used for testing the numerical equality of responses with base-type float. Note that this template always sets a value for SCORE, even if no RESPONSE was given.
  Full template URL: http://www.imsglobal.org/question/qtiv2p2/rptemplates/match_correct.xml
  
• MAP RESPONSE - the map response processing template uses the mapResponse operator to map the value of a response variable RESPONSE onto a value for the outcome SCORE. Note that when using the mapResponse, the SCORE needs to be of type float. Both variables must have been declared and RESPONSE must have an associated mapping. The template applies to responses of any baseType and cardinality. See the notes about mapResponse for details of its behaviour when applied to containers. If RESPONSE was NULL the SCORE is set to 0.0.
  Full template URL: http://www.imsglobal.org/question/qtiv2p2/rptemplates/map_response.xml
  
• MAP RESPONSE POINT - the map response point processing template uses the mapResponsePoint operator to map the value of a response variable RESPONSE onto a value for the outcome SCORE. Both variables must been declared and RESPONSE must have baseType point. See the notes about mapResponsePoint for details of its behaviour when applied to containers. If RESPONSE was NULL the SCORE is set to 0.
  Full template URL: http://www.imsglobal.org/question/qtiv2p2/rptemplates/map_response_point.xml

2.5.2. Generalized Response Resprocessing

The components for generalized response processing are shown in Figure 2.3.

• ResponseProcessing;
• ResponseRuleGroup (an abstract class);
• ResponseCondition;
• ResponseIf;
• ResponseElse;
• ResponseProcessingFragment.

2.6. Modal Feedback

Modal feedback is shown to the candidate directly following response processing. The value of an outcome variable is used in conjunction with the 'showHide' and 'identifier' attributes to determine whether or not the feedback is shown in a similar way to feedbackElement. The modal feedback is described using the classes:

• ModalFeedback.

2.7. Item Templates

Item templates are templates that can be used for producing large numbers of similar items. Such items are often called cloned items. Item templates can be used to produce items by special purpose Cloning Engines or, where delivery engines support them, be used directly to produce a dynamically chosen clone at the start of an item session.

Each item cloned from an item template is identical except for the values given to a set of template variables. An assessmentItem is therefore an item template if it contains one or more templateDeclarations and a set of templateProcessing rules for assigning them values.

A cloning engine that creates cloned items must assign a different identifier to each clone and record the values of the template variables used to create it. A report of an item session with such a clone can then be transformed into an equivalent report for the original item template by substituting the item template's identifier for the cloned item's identifier and adding the values of the template variables to the report. The class that supports template declaration is:

• TemplateDeclaration.

2.7.1. Using Template Variables in an Item's Body

Template variables can be referred to by printedVariable objects in the item body. The value of the template variable is used to create an appropriate run of text that is displayed. Template variables can also be used to conditionally control content through templateBlocks and templateInlines in a similar way to outcome variables with feedbackElements. Finally, template variables can be used to control the visibility of choices in interactions. The set of classes that support the use of template variables in an Item are:

• TemplateBlock;
• TemplateInline.

2.7.2. Using Template Variables in Operator Attributes Values

Some of the operators used in expressions have attributes to control their behaviour. The value of a template variable evaluated at the time the operator itself is evaluated, can be used as the value of some such attributes by using a reference to a template variable of the appropriate baseType and cardinality instead of a value from the value space of the attribute's target type. Attributes that support this type of attribute value substitution are declared with one of the following types.

When binding variable references as strings the variable's identifier must be enclosed in braces. For example, the string "{myVariable}" refers to the template variable with identifier myVariable. A references must match the identifier in a corresponding 'templateDeclaration'.

2.7.3. Template Processing

Template processing consists of one or more TemplateRules that are followed by the cloning engine or delivery system in order to assign values to the template variables. Template processing is identical in form to responseProcessing except that the purpose is to assign values to template variables, not outcome variables. The classes that provide template processing are:

• TemplateProcessing;
• TemplateRuleGroup (an abstract class);
• TemplateCondition;
• TemplateConstraint;
• TemplateIf;
• TemplateElse.

2.8. Tests, Testparts and Sections

A test is represented by the AssessmentTest class. A test is a group of assessmentItems with an associated set of rules that determine which of the items the candidate sees, in what order, and in what way the candidate interacts with them. The rules describe the valid paths through the test, when responses are submitted for response processing and when (if at all) feedback is to be given.

There are two constructs that are used to group togther Items: TestParts and Sections. A TestPart is used to divide a Test into parts that would normally be undertaken in separate test sessions. Whereas, a Section establishes groups of Items that have some common pedagogic testing objective. Sections are used to collect together Items which will then be presented to the learner. The order of presentation can be controlled using selection and ordering algorithms. A Test must have at lest one TestPart. A TestPart must have at least one Section. A Section may contain child Sections. A Section must contain at least one other Section or Item. The set of classes that realise the core test structures are:

• AssessmentTest;
• TestPart;
• AssessmentSection;
• AssessmentSectionRef;
• AssessmentItem;
• AssessmentItemRef.

Any Test, Section and/or Item or combination is exchanged as a '.zip' file. The format of this file is based upon the IMS Content Packaging v1.2 specification [CP, 12]. See the QTI ASI v2.2 XSD Binding specification [QTI, 21a] for further details on how content packaging is used.

2.8.1. Navigation and Submission

This specification defines two ways in which the overall behaviour of each test part can be controlled: the navigation mode and the submission mode (both of which are characteristics on the TestPart class).

The navigation mode determines the general paths that the candidate may take. A testPart in linear mode restricts the candidate to attempt each item in turn. Once the candidate moves on they are not permitted to return. A testPart in nonlinear mode removes this restriction - the candidate is free to navigate to any item in the test at any time. Test delivery systems are free to implement their own user interface elements to facilitate navigation provided they honour the navigation mode currently in effect. A test delivery system may implement nonlinear mode simply by providing a method to step forward or backwards through the test part.

The submission mode determines when the candidate's responses are submitted for response processing. A testPart in individual mode requires the candidate to submit their responses on an item-by-item basis. In simultaneous mode the candidate's responses are all submitted together at the end of the testPart.

The choice of submission mode determines the states through which each item's session can pass during the test. In simultaneous mode, response processing cannot take place until the testPart is complete so each item session passes between the interacting and suspended states only. By definition the candidate can take one and only one attempt at each item and feedback cannot be seen during the test. Whether or not the candidate can return to review their responses and/or any item-level feedback after the test, is outside the scope of this specification. Simultaneous mode is typical of paper-based tests.

In individual mode response processing may take place during the test and the item session may pass through any of the states described in Items, subject to the itemSessionControl settings in force. Care should be taken when designing user interfaces for systems that support nonlinear navigation mode in combination with individual submission. With this combination candidates may change their responses for an item and then leave it in the suspended state by navigating to a different item in the same part of the test. Test delivery systems need to make it clear to candidates that there are unsubmitted responses (akin to unsaved changes in a traditional document editing system) at the end of the test part. A test delivery system may force candidates to submit or discard such responses before moving to a different item in individual mode if this is more appropriate.

2.8.2. The Structure of a Test

For each test session, items and sub-sections are selected and arranged into order according to rules defined in the containing section. This process of selection and ordering defines a basic structure for each part of the test on a per-session basis. The paths that a candidate may take through this structure are then controlled by the mode settings for the test part and possibly by further preConditions or branchRules evaluated during the test session itself. Figure 2.4 illustrates part of a test and the way the items are structured into sections and sub-sections.

• Selection;
• Ordering.

2.8.3. Time Limits

In the context of a specific assessmentTest an item, or group of items, may be subject to a time constraint. This specification supports both minimum and maximum time constraints. The controlled time for a single item is simply the duration of the item session as defined by the builtin response variable duration. For assessmentSections, testParts and whole assessmentTests the time limits relate to the durations of all the item sessions plus any other time spent navigating that part of the test. In other words, the time includes time spent in states where no item is being interacted with, such as dedicated navigation screens.

The allowLateSubmission attribute regulates whether a candidate's response that is beyond the maxTime should still be accepted.

Minimum times are applicable to assessmentSections and assessmentItems only when linear navigation mode is in effect.

Delivery Engines are required to track and report the time spent on each test part when time limits are in force. If no time limit is in force for a given test part or section then the time spent may be tracked and reported but it is not required. Similarly, if no time limit is in force for an item that is not a Time Dependent Item then the time spent may be reported but is not required either.

The time spent on the test is recorded as if it were a built-in response variable called duration delcared at the test-level and of base-type duration and single cardinality. Similarly, time spent on test parts or sections are treated as built-in response variables declared within each respective scope. The values of these durations can be refered to during outcomeProcessing by using the variable name duration prefixed with the identifier of the part or section followed by the period character. See the variable expression for further information. The classes that realise the time limits are:

• TimeLimits.

2.9. Outcome Processing

Outcome processing takes place each time the candidate submits the responses for an item (when in individual submission mode) or a group of items (when in simultaneous submission mode). It happens after any (item level) response processing triggered by the submission. The values of the test's outcome variables are always reset to their defaults prior to carrying out the instructions described by the outcomeRules. Because outcome processing occurs each time the candidate submits responses the resulting values of the test-level outcomes may be used to activate test-level feedback during the test or to control the behaviour of subsequent parts through the use of preConditions and branchRules. The structure of outcome processing is similar to that or responseProcessing. The outcomes processing functionality is described in the classes:

• OutcomeProcessing;
• OutcomeRule;
• OutcomeCondition;
• OutcomeIf;
• OutcomeElse;
• OutcomeProcessingFragment.

2.10. Test-level Feedback

Test feedback is shown to the candidate either directly following outcome processing (during the test) or at the end of the testPart or assessmentTest as appropriate (referred to as atEnd). The core functionality of a test feedback is defined using the classes:

• TestFeedback.

2.11. Pre-conditions and Branching

2.11.1. Pre-conditions

Pre-conditions and branch rules are advanced concepts used to introduce an element of adaptivity into the specification of an assessmentTest. The concepts were introduced in version 1.2 (as pre-conditions and post-conditions) but their specification was postponed until version 2.1. A primary consideration in introducing these features has been to limit the complexity of supporting delivery engines while still enabling some items (or whole sections and test parts) to be skipped depending on the candidate's responses to items presented earlier in the test.

Pre-conditions and branch rules are only applicable to parts of the test that are navigated in linear mode. In nonlinear mode they are ignored. The overarching test parts are effectively navigated in nonlinear mode and so can therefore have associated pre-conditions and branch rules. The pre-condition functionality is provided through the attribute:

• Precondition (for AssessmentItemRef);
• Precondition (for AssessmentSection).

2.11.2. Branching

A branch-rule is a simple expression attached to an assessmentItemRef, assessmentSection or testPart that is evaluated after the item, section or part has been presented to the candidate. If the expression evaluates to true the test jumps forward to the item, section or part referred to by the target identifier. In the case of an item or section, the target must refer to an item or section in the same testPart that has not yet been presented. For testParts, the target must refer to another testPart. The branching functionality is provded in the class:

• BranchRule.

[Comment] The above definition restricts the navigation paths through a linear test part to being trees. In other words, cycles are not allowed. In most cases, repitition can be achieved by using a section that selects withReplacement up to a suitable upper bound of repitition in combination with a preCondition or branchRule that terminates the section early when (or if) a certain outcome has been achieved. This technique might be used in conjunction with one or more Item Templates to achieve drill and practice, for example. However, unbounded repetition is not supported. Comments are sought on whether this approach is too restrictive.

2.12. Expressions

QTI contains a wide range of expressions used to support the response processing, outcomes processing and template processing capabilities. Expressions are used to assign values to variables and to control conditional actions in response, outcomes and template processing. An expression can be a simple reference to the value of an item variable, a constant value from one of the value sets defined by baseTypes or a hierarchical expression operator. Like itemVariables, each expression can also have the special value NULL. The expressions are grouped into two abstract classes:

• ExpressionGroup - the full set of expressions;
• NumericGroupExpression - identifies the set of expression suited for numeric-bsed processing.

2.12.1. Builtin General Expressions

There are several built-in general expressions for handling constants, random values and the values of the varous item variables (response, outcome and template). These built-in expressions are realised using the classes:

• BaseValue - 'baseValue';
• Variable - 'variable';
• Default - 'default';
• Correct - 'correct';
• MapResponse -'mapResponse' and 'mapResponsePoint';
• MathConstant -'mathConstant'.

2.12.1.1. BaseValue Expression

The simplest expression returns a single value from the set defined by the given baseType.

2.12.1.2. Variable Expression

This expression looks up the value of an itemVariable that has been declared in a corresponding variableDeclaration or is one of the built-in variables. The result has the base-type and cardinality declared for the variable subject to the type promotion of weighted outcomes.

During outcomes processing, values taken from an individual item session can be looked up by prefixing the name of the item variable with the identifier assigned to the item in the assessmentItemRef, separated by a period character. For example, to obtain the value of the SCORE variable in the item referred to as Q01 you would use a variable instance with identifier Q01.SCORE.

In adaptive tests that contain items that are allowed to be replaced (i.e. that have the withReplacement attribute set to "true"), the same item can be instantiated more than once. In order to access the outcome variable values of each instantiation, a number that denotes the instance's place in the sequence of the item's instantiation is inserted between the item variable identifier and the item variable, separated by a period character. For example, to obtain the value of the SCORE variable in the item referred to as Q01 in its second instantiation you would use a variable instance, prefixed by the instantiation sequence number, prefixed by an identifier Q01.2.SCORE.

When looking up the value of a response variable it always takes the value assigned to it by the candidate's last submission. Unsubmitted responses are not available during expression evaluation.

The value of an item variable taken from an item instantiated multiple times from the same assessmentItemRef (through the use of selection withReplacement) is taken from the last instance submitted if submission is simultaneous, otherwise it is undefined.

2.12.1.3. Default Expression

This expression looks up the declaration of an itemVariable and returns the associated defaultValue or NULL if no default value was declared. When used in outcomes processing item identifier prefixing (see variable) may be used to obtain the default value from an individual item.

2.12.1.4. Correct Expression

This expression looks up the declaration of a response variable and returns the associated correctResponse or NULL if no correct value was declared. When used in outcomes processing item identifier prefixing (see variable) may be used to obtain the correct response from an individual item.

2.12.1.5. Map Response Exprssion

This expression looks up the value of a response variable and then transforms it using the associated mapping, which must have been declared. The result is a single float. If the response variable has single cardinality then the value returned is simply the mapped target value from the map. If the response variable has multiple or ordered cardinality then the value returned is the sum of the mapped target values. This expression cannot be applied to variables of record cardinality.

For example, if a mapping associates the identifiers {A,B,C,D} with the values {0,1,0.5,0} respectively then mapResponse will map the single value 'C' to the numeric value 0.5 and the set of values {C,B} to the value 1.5.

If a container contains multiple instances of the same value then that value is counted once only. To continue the example above {B,B,C} would still map to 1.5 and not 2.5.

2.12.1.6. Map Response Point Expression

This expression looks up the value of a response variable that must be of base-type point, and transforms it using the associated areaMapping. The transformation is similar to mapResponse except that the points are tested against each area in turn. When mapping containers each area can be mapped once only. For example, if the candidate identified two points that both fall in the same area then the mappedValue is still added to the calculated total just once.

2.12.1.7. Math Constant Expression

The result is a mathematical constant returned as a single float, e.g. Pi and e.

2.12.2. Expressions Used only in Outcomes Processing

There are several expressions for handling outcomes processing only. These expressions are realised using the classes:

• Number - 'numberCorrect, 'numberIncorrect', 'numberPresented', 'numberResponded' and 'numberSelected';
• OutcomeMinMax - 'outcomeMinimum' and 'outcomeMaximum';
• TestVariables - 'testVariables'.

2.12.2.1. Number Correct Expression

This expression, which can only be used in outcomes processing, calculates the number of items in a given sub-set, for which the all defined response variables match their associated correctResponse. Only items for which all declared response variables have correct responses defined are considered. The result is an integer with single cardinality.

2.12.2.2. Number Incorrect Expression

This expression, which can only be used in outcomes processing, calculates the number of items in a given sub-set, for which at least one of the defined response variables does not match its associated correctResponse. Only items for which all declared response variables have correct responses defined and have been attempted at least once are considered. The result is an integer with single cardinality.

2.12.2.3. Number Presented Expression

This expression, which can only be used in outcomes processing, calculates the number of items in a given sub-set that have been attempted (at least once). In other words, items with which the user has interacted, whether or not they provided a response. The result is an integer with single cardinality.

2.12.2.4. Number Responded Expresson

This expression, which can only be used in outcomes processing, calculates the number of items in a given sub-set that have been attempted (at least once) and for which a response was given. In other words, items for which at least one declared response has a value that differs from its declared default (typically NULL). The result is an integer with single cardinality.

2.12.2.5. Number Selected Expression

This expression, which can only be used in outcomes processing, calculates the number of items in a given sub-set that have been selected for presentation to the candidate, regardless of whether the candidate has attempted them or not. The result is an integer with single cardinality.

2.12.2.6. Outcome Minimum Expression

2.12.2.7. Outcome Maximum Expression

This expression, which can only be used in outcomes processing, simultaneously looks up the normalMaximum value of an outcome variable in a sub-set of the items referred to in a test. Only variables with single cardinality are considered. If any of the items within the given subset have no declared maximum the result is NULL, otherwise the result has cardinality multiple and base-type float.

2.12.2.8. Test Variables Expression

This expression, which can only be used in outcomes processing, simultaneously looks up the value of an itemVariable in a sub-set of the items referred to in a test. Only variables with single cardinality are considered, all NULL values are ignored. The result has cardinality multiple and base-type as specified below.

2.12.3. Operators

Operators are a family of classes derived from expression that obtain their value (referred to as their result) either by modifying a single sub-expression or by combining two or more sub-expressions in a specified way. Operators never effect the values of itemVariables directly, in other words, there are no 'side effects'. All operators have a baseType and cardinality though these may be dependent on the sub-expression(s) they contain. The set of classes that realise these operators are:

• AnyN - 'anyN';
• CustomOperator - 'customOperator';
• Equal - 'equal';
• EqualRounded - 'equalRounded';
• FieldValue -'fieldValue';
• Index -'index';
• Inside - 'inside';
• Logic0toMany - 'multiple' and 'ordered';
• Logic1toMany - 'and', 'gcd', 'lcm', 'min', 'max', 'or' and 'product';
• LogicPair - 'contains', 'delete', 'divide', 'durationgte', 'durationlt', 'gt', 'gte', 'integerDivide', 'integerModulus', 'lt', 'lte', 'match', 'member', 'power' and 'subtract';
• LogicSingle - 'containerSize', 'integerToFloat', 'isNull', 'not', 'random', 'round' and 'truncate';
• MathOperator - 'mathOperator';
• NumericLogic1toMany - 'sum';
• PatternMatch -'patternMatch';
• Repeat -'repeat';
• RoundTo - 'roundTo';
• StatsOperator - 'statsOperator';
• StringMatch -'stringMatch';
• Substring - 'substring'.

2.12.3.1. AnyN Expression

The anyN operator takes one or more sub-expressions each with a base-type of boolean and single cardinality. The result is a single boolean which is true if at least min of the sub-expressions are true and at most max of the sub-expressions are true. If more than n - min sub-expressions are false (where n is the total number of sub-expressions) or more than max sub-expressions are true then the result is false. If one or more sub-expressions are NULL then it is possible that neither of these conditions is satisfied, in which case the operator results in NULL. For example, if min is 3 and max is 4 and the sub-expressions have values {true,true,false,NULL} then the operator results in NULL whereas {true,false,false,NULL} results in false and {true,true,true,NULL} results in true. The result NULL indicates that the correct value for the operator cannot be determined.

2.12.3.2. Custom Operator Expression

The custom operator provides an extension mechanism for defining operations not currently supported by this specification.

2.12.3.3. Equal Expression

The equal operator takes two sub-expressions which must both have single cardinality and have a numerical base-type. The result is a single boolean with a value of true if the two expressions are numerically equal and false if they are not. If either sub-expression is NULL then the operator results in NULL.

2.12.3.4. Equal Rounded Expression

The equalRounded operator takes two sub-expressions which must both have single cardinality and have a numerical base-type. The result is a single boolean with a value of true if the two expressions are numerically equal after rounding and false if they are not. If either sub-expression is NULL then the operator results in NULL.

2.12.3.5. Field Value Expression

The identifier of the field to be selected. The field-value operator takes a sub-expression with a record container value. The result is the value of the field with the specified fieldIdentifier. If there is no field with that identifier then the result of the operator is NULL.

2.12.3.6. Index Expression

The index operator takes a sub-expression with an ordered container value and any base-type. The result is the nth value of the container. The result has the same base-type as the sub-expression but single cardinality. The first value of a container has index 1, the second 2 and so on. n must be a positive integer. If n exceeds the number of values in the container (or the sub-expression is NULL) then the result of the index operator is NULL. If n is an identifier, it is the value of n at runtime that is used.

2.12.3.7. Inside Expression

The inside operator takes a single sub-expression which must have a baseType of point. The result is a single boolean with a value of true if the given point is inside the area defined by shape and coords. If the sub-expression is a container the result is true if any of the points are inside the area. If either sub-expression is NULL then the operator results in NULL.

2.12.3.8. Multiple Expression

The multiple operator takes 0 or more sub-expressions all of which must have either single or multiple cardinality. Although the sub-expressions may be of any base-type they must all be of the same base-type. The result is a container with multiple cardinality containing the values of the sub-expressions, sub-expressions with multiple cardinality have their individual values added to the result: containers cannot contain other containers. For example, when applied to A, B and {C,D} the multiple operator results in {A,B,C,D}. All sub-expressions with NULL values are ignored. If no sub-expressions are given (or all are NULL) then the result is NULL.

2.12.3.9. Ordered Expression

The ordered operator takes 0 or more sub-expressions all of which must have either single or ordered cardinality. Although the sub-expressions may be of any base-type they must all be of the same base-type. The result is a container with ordered cardinality containing the values of the sub-expressions, sub-expressions with ordered cardinality have their individual values added (in order) to the result: contains cannot contain other containers. For example, when applied to A, B, {C,D} the ordered operator results in {A,B,C,D}. Note that the ordered operator never results in an empty container. All sub-expressions with NULL values are ignored. If no sub-expressions are given (or all are NULL) then the result is NULL.

2.12.3.10. And Expression

The and operator takes one or more sub-expressions each with a base-type of boolean and single cardinality. The result is a single boolean which is true if all sub-expressions are true and false if any of them are false. If one or more sub-expressions are NULL and all others are true then the operator also results in NULL.

2.12.3.11. GCD Expression

The gcd operator takes 1 or more sub-expressions which all have base-type integer and may have single, multiple or ordered cardinality. The result is a single integer equal in value to the greatest common divisor (gcd) of the argument values. If all the arguments are zero, the result is 0, gcd(0,0)=0; authors should beware of this in calculations which require division by the gcd of random values. If some, but not all, of the arguments are zero, the result is the gcd of the non-zero arguments, gcd(0,n)=n if n!=0. If any of the sub-expressions is NULL, the result is NULL. If any of the sub-expressions is not a numerical value, then the result is NULL.

2.12.3.12. LCM Expression

The lcm operator takes 1 or more sub-expressions which all have base-type integer and may have single, multiple or ordered cardinality. The result is a single integer equal in value to the lowest common multiple (lcm) of the argument values. If any argument is zero, the result is 0, lcm(0,n)=0; authors should beware of this in calculations which require division by the lcm of random values. If any of the sub-expressions is NULL, the result is NULL. If any of the sub-expressions is not a numerical value, then the result is NULL.

2.12.3.13. Min Expression

The min operator takes 1 or more sub-expressions which all have numerical base-types and may have single, multiple or ordered cardinality. The result is a single float, or, if all sub-expressions are of integer type, a single integer, equal in value to the smallest of the argument values, i.e. the result is the argument closest to negative infinity. If the arguments have the same value, the result is that same value. If any of the sub-expressions is NULL, the result is NULL. If any of the sub-expressions is not a numerical value, then the result is NULL.

2.12.3.14. Max Expression

The max operator takes 1 or more sub-expressions which all have numerical base-types and may have single, multiple or ordered cardinality. The result is a single float, or, if all sub-expressions are of integer type, a single integer, equal in value to the greatest of the argument values, i.e. the result is the argument closest to positive infinity. If the arguments have the same value, the result is that same value. If any of the sub-expressions is NULL, the result is NULL. If any of the sub-expressions is not a numerical value, then the result is NULL.

2.12.3.15. Or Expression

The or operator takes one or more sub-expressions each with a base-type of boolean and single cardinality. The result is a single boolean which is true if any of the sub-expressions are true and false if all of them are false. If one or more sub-expressions are NULL and all the others are false then the operator also results in NULL.

2.12.3.16. Product Expression

The product operator takes 1 or more sub-expressions which all have numerical base-types and may have single, multiple or ordered cardinality. The result is a single float or, if all sub-expressions are of integer type, a single integer that corresponds to the product of the numerical values of the sub-expressions. If any of the sub-expressions are NULL then the operator results in NULL.

2.12.3.17. Contains Expression

The contains operator takes two sub-expressions which must both have the same base-type and cardinality - either multiple or ordered. The result is a single boolean with a value of true if the container given by the first sub-expression contains the value given by the second sub-expression and false if it doesn't. Note that the contains operator works differently depending on the cardinality of the two sub-expressions. For unordered containers the values are compared without regard for ordering, for example, [A,B,C] contains [C,A]. Note that [A,B,C] does not contain [B,B] but that [A,B,B,C] does. For ordered containers the second sub-expression must be a strict sub-sequence within the first. In other words, [A,B,C] does not contain [C,A] but it does contain [B,C].

If either sub-expression is NULL then the result of the operator is NULL. Like the member operator, the contains operator should not be used on sub-expressions with a base-type of float and must not be used on sub-expressions with a base-type of duration.

2.12.3.18. Delete Expression

The delete operator takes two sub-expressions which must both have the same base-type. The first sub-expression must have single cardinality and the second must be a multiple or ordered container. The result is a new container derived from the second sub-expression with all instances of the first sub-expression removed. For example, when applied to A and {B,A,C,A} the result is the container {B,C}. If either sub-expression is NULL the result of the operator is NULL. The restrictions that apply to the member operator also apply to the delete operator.

2.12.3.19. Divide Expression

The divide operator takes 2 sub-expressions which both have single cardinality and numerical base-types. The result is a single float that corresponds to the first expression divided by the second expression. If either of the sub-expressions is NULL then the operator results in NULL.

Item authors should make every effort to ensure that the value of the second expression is never 0, however, if it is zero or the resulting value is outside the value set defined by float (not including positive and negative infinity) then the operator should result in NULL.

2.12.3.20. Duration Expression

The durationGTE operator takes two sub-expressions which must both have single cardinality and base-type duration. The result is a single boolean with a value of true if the first duration is longer (or equal, within the limits imposed by truncation as described above) than the second and false if it is shorter than the second. If either sub-expression is NULL then the operator results in NULL. See durationLT for more information about testing the equality of durations.

2.12.3.21. Duration LT Expression

The durationLT operator takes two sub-expressions which must both have single cardinality and base-type duration. The result is a single boolean with a value of true if the first duration is shorter than the second and false if it is longer than (or equal) to the second. If either sub-expression is NULL then the operator results in NULL.

There is no 'durationLTE' or 'durationGT' because equality of duration is meaningless given the variable precision allowed by duration. Given that duration values are obtained by truncation rather than rounding it makes sense to test only less-than or greater-than-equal inequalities only. For example, if we want to determine if a candidate took less than 10 seconds to complete a task in a system that reports durations to a resolution of epsilon seconds (epsilon<1) then a value equal to 10 would cover all durations in the range [10,10+epsilon).

2.12.3.22. GT Expression

The gt operator takes two sub-expressions which must both have single cardinality and have a numerical base-type. The result is a single boolean with a value of true if the first expression is numerically greater than the second and false if it is less than or equal to the second. If either sub-expression is NULL then the operator results in NU

2.12.3.23. GTE Expression

The gte operator takes two sub-expressions which must both have single cardinality and have a numerical base-type. The result is a single boolean with a value of true if the first expression is numerically greater than or equal to the second and false if it is less than the second. If either sub-expression is NULL then the operator results in NULL.

2.12.3.24. Integer Divide Expression

The integer divide operator takes 2 sub-expressions which both have single cardinality and base-type integer. The result is the single integer that corresponds to the first expression (x) divided by the second expression (y) rounded down to the greatest integer (i) such that i LTE (x/y). If y is 0, or if either of the sub-expressions is NULL then the operator results in NULL.

2.12.3.25. Integer Modulus Expression

The integer modulus operator takes 2 sub-expressions which both have single cardinality and base-type integer. The result is the single integer that corresponds to the remainder when the first expression (x) is divided by the second expression (y). If z is the result of the corresponding integerDivide operator then the result is x-z*y. If y is 0, or if either of the sub-expressions is NULL then the operator results in NULL.

2.12.3.26. LT Expression

The lt operator takes two sub-expressions which must both have single cardinality and have a numerical base-type. The result is a single boolean with a value of true if the first expression is numerically less than the second and false if it is greater than or equal to the second. If either sub-expression is NULL then the operator results in NULL.

2.12.3.27. LTE Expression

The lte operator takes two sub-expressions which must both have single cardinality and have a numerical base-type. The result is a single boolean with a value of true if the first expression is numerically less than or equal to the second and false if it is greater than the second. If either sub-expression is NULL then the operator results in NULL.

2.12.3.28. Match Expression

The match operator takes two sub-expressions which must both have the same base-type and cardinality. The result is a single boolean with a value of true if the two expressions represent the same value and false if they do not. If either sub-expression is NULL then the operator results in NULL.

The match operator must not be confused with broader notions of equality such as numerical equality. To avoid confusion, the match operator should not be used to compare subexpressions with base-types of float and must not be used on sub-expressions with a base-type of duration.

2.12.3.29. Member Expression

The member operator takes two sub-expressions which must both have the same base-type. The first sub-expression must have single cardinality and the second must be a multiple or ordered container. The result is a single boolean with a value of true if the value given by the first sub-expression is in the container defined by the second sub-expression. If either sub-expression is NULL then the result of the operator is NULL. The member operator should not be used on sub-expressions with a base-type of float because of the poorly defined comparison of values. It must not be used on sub-expressions with a base-type of duration.

2.12.3.30. Power Expression

The power operator takes 2 sub-expression which both have single cardinality and numerical base-types. The result is a single float that corresponds to the first expression raised to the power of the second. If either or the sub-expressions is NULL then the operator results in NULL. If the resulting value is outside the value set defined by float (not including positive and negative infinity) then the operator shall result in NULL.

2.12.3.31. Subcontract Expression

The subtract operator takes 2 sub-expressions which all have single cardinality and numerical base-types. The result is a single float or, if both sub-expressions are of integer type, a single integer that corresponds to the first value minus the second. If either of the sub-expressions is NULL then the operator results in NULL.

2.12.3.32. Container Expression

The containerSize operator takes a sub-expression with any base-type and either multiple or ordered cardinality. The result is an integer giving the number of values in the sub-expression, in other words, the size of the container. If the sub-expression is NULL the result is 0. This operator can be used for determining how many choices were selected in a multiple-response choiceInteraction, for example.

2.12.3.33. Integer to Float Expression

The integer to float conversion operator takes a single sub-expression which must have single cardinality and base-type integer. The result is a value of base type float with the same numeric value. If the sub-expression is NULL then the operator results in NULL.

2.12.3.34. IsNull Expression

The isNull operator takes a sub-expression with any base-type and cardinality. The result is a single boolean with a value of true if the sub-expression is NULL and false otherwise. Note that empty containers and empty strings are both treated as NULL.

2.12.3.35. Not Expression

The not operator takes a single sub-expression with a base-type of boolean and single cardinality. The result is a single boolean with a value obtained by the logical negation of the sub-expression's value. If the sub-expression is NULL then the not operator also results in NULL.

2.12.3.36. Random Expression

The random operator takes a sub-expression with a multiple or ordered container value and any base-type. The result is a single value randomly selected from the container. The result has the same base-type as the sub-expression but single cardinality. If the sub-expression is NULL then the result is also NULL.

2.12.3.37. Round Expression

The round operator takes a single sub-expression which must have single cardinality and a numerical base-type. The result is a value of base-type integer formed by rounding the value of the sub-expression. The result is the integer n for all input values in the range [n-0.5,n+0.5). In other words, 6.8 and 6.5 both round up to 7, 6.49 rounds down to 6 and -6.5 rounds up to -6. If the sub-expression is NULL then the operator results in NULL. If the sub-expression is NaN, then the result is NULL. If the sub-expression is INF, then the result is INF. If the sub-expression is -INF, then the result is -INF.

2.12.3.38. Truncate Expression

The truncate operator takes a single sub-expression which must have single cardinality and a numerical base-type. The result is a value of base-type integer formed by truncating the value of the sub-expression towards zero. For example, the value 6.8 becomes 6 and the value -6.8 becomes -6. If the sub-expression is NULL then the operator results in NULL. If the sub-expression is NaN, then the result is NULL. If the sub-expression is INF, then the result is INF. If the sub-expression is -INF, then the result is -INF.

2.12.3.39. Math Operator Expression

The mathOperator operator takes 1 or more sub-expressions which all have single cardinality and have numerical base-types. The trigonometric functions, sin, cos and tan, take one argument in radians, which evaluates to a single float. Other functions take one numerical argument. Further functions might take more than one numerical argument, e.g. atan2 (two argument arc tan). The result is a single float, except for the functions signum, floor and ceil, which return a single integer. If any of the sub-expressions is NULL, the result is NULL. If any of the sub-expressions falls outside the natural domain of the function called by mathOperator, e.g. log(0) or asin(2), then the result is NULL.

The reciprocal trigonometric functions also follow these rules:

• If the argument is NaN, then the result is NULL;
• If the value of tan for the argument is INF, then the value of cot is 0;
• If the value of tan for the argument is -INF, then the value of cot is 0;
• If the value of a trigonometric function is 0, then the value of the corresponding reciprocal trigonometric function is NULL.

The reciprocal trigonometric and hyperbolic functions also follow these rules:

• If the argument is NaN, then the result is NULL;
• If the value of a trigonometric or hyperbolic function for the argument is INF, then the value of the corresponding reciprocal trigonometric or hyperbolic function is 0;
• If the value of a trigonometric or hyperbolic function for the argument is -INF, then the value of the corresponding reciprocal trigonometric or hyperbolic function is 0;
• If the value of a trigonometric or hyperbolic function for the argument is 0, then the value of the corresponding reciprocal trigonometric or hyperbolic function is NULL;
• If the value of a trigonometric or hyperbolic function for the argument is -0, then the value of the corresponding reciprocal trigonometric or hyperbolic function is NULL.

The function atan2 also follows these rules:

• If either argument is NaN, then the result is NULL;
• If the first argument is positive zero and the second argument is positive, or the first argument is positive and finite and the second argument is positive infinity, then the result is 0;
• If the first argument is negative zero and the second argument is positive, or the first argument is negative and finite and the second argument is positive infinity, then the result is 0
• If the first argument is positive zero and the second argument is negative, or the first argument is positive and finite and the second argument is negative infinity, then the result is the double value closest to Pi
• If the first argument is negative zero and the second argument is negative, or the first argument is negative and finite and the second argument is negative infinity, then the result is the double value closest to -Pi;
• If the first argument is positive and the second argument is positive zero or negative zero, or the first argument is positive infinity and the second argument is finite, then the result is the double value closest to Pi/2;
• If the first argument is negative and the second argument is positive zero or negative zero, or the first argument is negative infinity and the second argument is finite, then the result is the double value closest to -Pi/2;
• If both arguments are positive infinity, then the result is the double value closest to Pi/4;
• If the first argument is positive infinity and the second argument is negative infinity, then the result is the double value closest to 3*Pi/4;
• If the first argument is negative infinity and the second argument is positive infinity, then the result is the double value closest to -Pi/4;
• If both arguments are negative infinity, then the result is the double value closest to -3*Pi/4.

2.12.3.40. Sum Expression

The sum operator takes 1 or more sub-expressions which all have numerical base-types and may have single, multiple or ordered cardinality. The result is a single float or, if all sub-expressions are of integer type, a single integer that corresponds to the sum of the numerical values of the sub-expressions. If any of the sub-expressions are NULL then the operator results in NULL.

2.12.3.41. Pattern Match Expression

The patternMatch operator takes a sub-expression which must have single cardinality and a base-type of string. The result is a single boolean with a value of true if the sub-expression matches the regular expression given by pattern and false if it doesn't. If the sub-expression is NULL then the operator results in NULL.

2.12.3.42. Repeat Expression

The repeat operator takes 0 or more sub-expressions, all of which must have either single or ordered cardinality and the same baseType. The result is an ordered container having the same baseType as its sub-expressions. The container is filled sequentially by evaluating each sub-expression in turn and adding the resulting single values to the container, iterating this process numberRepeats times in total. If numberRepeats refers to a variable whose value is less than 1, the value of the whole expression is NULL. Any sub-expressions evaluating to NULL are ignored. If all sub-expressions are NULL then the result is NULL.

2.12.3.43. RoundTo Expression

The roundTo operator takes one sub-expression which must have single cardinality and a numerical base-type. The result is a single float with the value nearest to that of the expression's value such that when converted to a decimal string it represents the expression rounded by the specified rounding method to the specified precision. If the sub-expression is NULL, then the result is NULL. If the sub-expression is INF, then the result is INF. If the sub-expression is -INF, then the result is -INF. If the argument is NaN, then the result is NULL.

When rounding to n significant figures, the deciding digit is the (n+1)th digit counting from the first non-zero digit from the left in the number. If the deciding digit is 5 or greater, the nth digit is increased by 1 and all digits to its right are discarded; if the deciding digit is less than 5, all digits to the right of the nth digit are discarded.

When rounding to n decimal places, the deciding digit is the (n+1)th digit counting to the right from the decimal point. If the deciding digit is 5 or greater, the nth digit is increased by 1 and all digits to its right are discarded; if the deciding digit is less than 5, all digits to the right of the nth digit are discarded.

2.12.3.44. Stats Operator Expression

The statsOperator operator takes 1 sub-expression which is a container of multiple or ordered cardinality and has a numerical base-type. The result is a single float. If the sub-expression or any value contained therein is NULL, the result is NULL. If any value contained in the sub-expression is not a numerical value, then the result is NULL.

2.12.3.45. String Match Expression

The stringMatch operator takes two sub-expressions which must have single and a base-type of string. The result is a single boolean with a value of true if the two strings match according to the comparison rules defined by the attributes below and false if they don't. If either sub-expression is NULL then the operator results in NULL.

2.12.3.46. Substring Expression

The substring operator takes two sub-expressions which must both have an effective base-type of string and single cardinality. The result is a single boolean with a value of true if the first expression is a substring of the second expression and false if it isn't. If either sub-expression is NULL then the result of the operator is NULL.

2.13. Items and Test Fragments

An Item Fragment is part of an item that is managed independently of the items that depend on it. Similarly, a Test Fragment is part of a test that is managed independently of the tests that depend on it. Fragments are packaged as separate resources and can be transported independently. A fragment may appear anywhere in the model where one of the following abstract classes may appear: flowStatic, inlineStatic, blockStatic, responseRule, sectionPart or outcomeRule. For example, an item fragment may be a division of the itemBody represented by an instance of the 'Div' class or a single interaction.

Fragments are included using the Include mechanism (the actual form of implementation is dependent on the binding approach). The instance of include is treated as if it was actually an instance of the root element of the fragment referred to by the 'href' chracteristic of the 'Include' class. For the purposes of this specification, when using an XML binding, the xpointer mechanism defined by the XInclude specification must not be used. Also, all included fragments must be treated as parsed xml.

This technique is similar to the inclusion of media objects (using object) but allows the inclusion of data that conforms to this specification, specifically, it allows the inclusion of interactions, static content, processing rules or, at test level whole sections, to be included from externally defined fragments.

When including externally defined fragments the content of the fragment must satisfy the requirements of the specification in the context of the item in which it is being included. For example, interactions included from fragments must be correctly bound to response variables declared in the items. Item and Test fragments are supported using the classes:

• Include (an imported class);
• ResponseProcessingFragment;
• OutcomeProcessingFragment.

2.14. The Accessibility Features

There are three accessibility features that are supported by QTI 2.2:

• Accessible Portable Item Protocol (APIP) - enables the APIP markup to be used;
• Speech Synthesis Markup Language (SSML) - enables text to have SSML annotation
• WAI-ARIA - allows the ARIA annotations to be used on the HTML and interaction tags

2.14.1. APIP Accessibility Functionality

The Accessible Portable Item Protocol (APIP) specification provides a rich set of accessibility annotation capabilities to the core QTI ASI functionality [APIP, 14]. There are five components to the APIP annotation:

• Item Information - this is the metadata that accompanies the Item. This could include information about versioning, subject domain, relevant curriculum standard learning objectives addressed, etc.
• Content Information - information about the contents of the item that are to be presented to an examinee assuming no access needs have been defined for that user;
• Companion Material - content props that provide key information to be used when answering an Item, e.g. a calculator, protractor, lookup chart, map, reading passage, etc.
• Accessibility Information - the alternative content to be used to render the content in its described accessibility forms, e.g., read aloud, Braille, sign language, etc.
• Inclusion Order - the rendering order of the content to be presented to the user undertaking the assessment. The actual order is determined by the user's accessibility preferences and the range of alternative rendering options created for the Item.

APIP functionality is provided through the APIPAccessibility class that is imported. APIP anotations, for QTI ASI these are always optional content , are permitted on the following classes:

• AssessmentItem;
• AssessmentStimulus;
• FeedbackBlock;
• RubricBlock;
• RubricBlockTemplateBlock;
• TemplateBlock;
• TemplateBlockFeedbackBlock;
• TestFeedback.

Further information on on how to apply, and the consequences of, APIP annotation is available in [APIP, 14]. An example of how to use APIP is given in the QTI Best Practices document [QTI, 15].

2.14.2. Support for Speech Synthesis Markup Language

The Speech Synthesis Markup Language Specification is designed to provide a rich, XML-based markup language for assisting the generation of synthetic speech in Web and other applications. The essential role of the markup language is to provide authors of synthesizable content a standard way to control aspects of speech such as pronunciation, volume, pitch, rate, etc. across different synthesis-capable platforms.

SSML functionality is provided through the SSMLGroup class that is imported. The SSML tags that are supported are 'p', 's', 'say-as', 'phoneme', 'sub', 'voice', 'emphasis', 'break', 'prosody' 'mark', 'audio' and 'speak'. SSML anotations are supplied through the following classes:

• FlowContentModel (an abstract class);
• InlineContentModel (an abstract class).

Further information on SSMLv1.1 is available in [SSML, 10]. An example of how to use SSML is given in the QTI Best Practices document [QTI, 15].

2.14.3. Support for WAI-ARIA

The Accessible Rich Internet Applications (ARIA) specification from W3C [ARIA, 14] provides an ontology of roles, states, and properties that define accessible user interface elements and can be used to improve the accessibility and interoperability of web content and applications. These semantics are designed to allow an author to properly convey user interface behaviors and structural information to assistive technologies in document-level markup. In QTI the ARIA characteristics are used on the following classes:

• BasePromptInteraction;
• BaseSequence;
• BaseSequenceFull;
• BaseSequenceRIdent;
• BaseSequenceXBase;
• BaseSequenceXBaseEmpty.

These classes are used to define the HTML4 and the QTI Interactions. The ARIA features are also supported on the HTML5 tags [QTI, 21b]. The ARIA features that are supported, this is a subset of the possible set of ARIA characteristics, in QTI are:

• role;
• aria-controls;
• aria-describedby;
• aria-flowto;
• aria-label;
• aria-labelledby;
• aria-level;
• aria-orientation;
• aria-owns.

Further information on WAI-ARIA 1.0 is available in [ARIA, 14]. An example of how to use ARIA is given in the QTI Best Practices document [QTI, 15].

2.15. The Internationalization Features

As part of the QTIv2.2 new features, support for internationalization was added. The internationalization features now supported are:

• Bidirectional Text - provision of right-to-left and left-to-right text presentation;
• Ruby - these annotations are short runs of text presented alongside base text, primarily used in East Asian typography as a guide for pronunciation or to include other annotations;
• Vertical Writing - this is achieved by using CSS as recommended in the W3C's 'CSS Writing Modes Module Level 3' [W3C, 14] (at the time of QTI publication this W3C document was a Working Draft).

2.15.1. Support for Bidirectional Text

For different languages the text is presented and read either from 'left-to-right' or 'right-to-left'. The corresponding rendering must take into account word and character spacing and justification. The classes that enable the bi-directional text layout support are:

• BDO;
• BDI - this is part of the new HTML5 functionality, see [QTI, 21b].

2.15.2. Support for Ruby

Ruby allows one or more spans of phrasing content to be marked with ruby annotations. Ruby annotations are short runs of text presented alongside base text, primarily used in East Asian typography as a guide for pronunciation or to include other annotations. In Japanese, this form of typography is also known as furigana. Ruby text can appear on either side, and sometimes both sides, of the base text, and it is possible to control its position using CSS. A more complete introduction to ruby can be found in the Use Cases and Exploratory Approaches for Ruby Markup document as well as in CSS Ruby Module Level 1. [RUBYUC, 13] [CSSRUBY, 15]. The classes that enable the Ruby support are:

• Ruby - this is part of the new HTML5 functionality, see [QTI, 21b];
• RB - this is part of the new HTML5 functionality, see [QTI, 21b];
• RP- this is part of the new HTML5 functionality, see [QTI, 21b];
• RT - this is part of the new HTML5 functionality, see [QTI, 21b];
• RTC - this is part of the new HTML5 functionality, see [QTI, 21b];
• RPRTSelection - abstract class and this is part of the new HTML5 functionality, see [QTI, 21b];
• RTRTCSelection - abstract class and this is part of the new HTML5 functionality, see [QTI, 21b];
• RTubySelection - abstract class and this is part of the new HTML5 functionality, see [QTI, 21b].

3. Root Attribute Descriptions

All of the Root attributes (the root name for the instances that can be exchanged) used within this Information Model are described in this Section. The syntax and semantics for this representation is described in Appendix A1.3. The root attributes are:

• assessmentItem - this is the root element for the exchange of items. The assessmentItem may reference any required content stimuli contained within an assessmentStimulus. The exchange of a single assessmentItem instance is permitted;

• assessmentSection - this is the root element for the exchange of sections. An assessmentSection can contain any order and hierarchy of contained/referenced assessmentSections and referenced assessmentItems. The assessmentItems can only be referenced from within an assessmentSection i.e. they cannot be contained in the same instance as the assessmentSection. The exchange of a single root assessmentSection instance is permitted but any number of contained assessmentSections can also be exchanged;

• assessmentStimulus - this is the root element for the exchange of stimuli i.e. content that is to be shared and referenced by multiple items. This allows the stimulus content to be managed as a separate first class object. The exchange of a single assessmentStimulus instance is permitted;

• assessmentTest - this is the root element for the exchange of tests, quizzes, etc. The test must contain at least one test part which in turn must contain, or reference, at least one assessmentSection. The associated assessmentItems can only be referenced from within an assessmentSection and cannot be contained in the same instance as the assessmentTest. The exchange of a single assessmentTest instance is permitted;

• outcomeDeclaration - this is the root element for the exchange of outcomes declarations. These declarations are typically used to identify the relationship between an outcome and the associated curriculum standard i.e. to indicate which outcomes support specific learning objectives. The exchange of a single outcome declaration instance is permitted;

• responseProcessing - this is the root element for the exchange of response processing templates. Response processing templates are used to define generic templates that can be applied to response processing for a range of types of assessmentItems. The exchange of a single response processing template instance is permitted.

3.1 "assessmentItem" Root Attribute Description

3.2 "assessmentSection" Root Attribute Description

3.3 "assessmentStimulus" Root Attribute Description

3.4 "assessmentTest" Root Attribute Description

3.5 "outcomeDeclaration" Root Attribute Description

3.6 "responseProcessing" Root Attribute Description

4. Root Class Descriptions

All of the Root data classes (the first class objects that can be exchanged using the data model) used within this Information Model are described in this Section. The syntax and semantics for this representation is described in Appendix A1.2.

4.1 AssessmentItem Root Class Description

4.1.1 "identifier" Characteristic Description