1EdTech Question & Test Interoperability: Results Reporting Information Model Final Specification Version 1.2

Copyright © 2002 1EdTech Consortium, Inc. All Rights Reserved. The 1EdTech Logo is a trademark of 1EdTech Consortium, Inc. Document Name: 1EdTech Question & Test Interoperability: Results Reporting Information Model Date: 11 February 2002

IPR and Distribution Notices

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the specification set forth in this document, and to provide supporting documentation.

1EdTech takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on 1EdTech's procedures with respect to rights in 1EdTech specifications can be found at the 1EdTech Intellectual Property Rights web page: http://www.imsglobal.org/ipr/imsipr_policyFinal.pdf.

Copyright © 2002 1EdTech Consortium. All Rights Reserved.

Permission is granted to all parties to use excerpts from this document as needed in producing requests for proposals.

Use of this specification to develop products or services is governed by the license with 1EdTech found on the 1EdTech website: http://www.imsglobal.org/license.html.

The limited permissions granted above are perpetual and will not be revoked by 1EdTech or its successors or assigns.

THIS SPECIFICATION IS BEING OFFERED WITHOUT ANY WARRANTY WHATSOEVER, AND IN PARTICULAR, ANY WARRANTY OF NONINFRINGEMENT IS EXPRESSLY DISCLAIMED. ANY USE OF THIS SPECIFICATION SHALL BE MADE ENTIRELY AT THE IMPLEMENTER'S OWN RISK, AND NEITHER THE CONSORTIUM, NOR ANY OF ITS MEMBERS OR SUBMITTERS, SHALL HAVE ANY LIABILITY WHATSOEVER TO ANY IMPLEMENTER OR THIRD PARTY FOR ANY DAMAGES OF ANY NATURE WHATSOEVER, DIRECTLY OR INDIRECTLY, ARISING FROM THE USE OF THIS SPECIFICATION.

---

 

Table of Contents

1. Introduction
     1.1 Question & Test Interoperability Overview
     1.2 Scope & Context
     1.3 Structure of this Document
     1.4 Nomenclature
     1.5 References

2. Specification Use-Cases
     2.1 Basic Architecture
     2.2 Results Aggregation System
     2.3 Processed Results for a Learning Management System
     2.4 Real-time Interactive
     2.5 Internal Interchange Data Language Representation

3. Basic Information Model

4. Conceptual Description of the Data Objects
     4.1 Underlying Structure of the Results Report Package
     4.2 Extensions & Extensibility
     4.3 QTI Results Report Package Tabular Description
           4.3.1 QTI Results Reporting Data Objects
           4.3.2 'summary_result' Data Objects
           4.3.3 'assessment_result' Data Objects
           4.3.4 'section_result' Data Objects
           4.3.5 'item_result' Data Objects
           4.3.6 Common Data Objects
           4.3.7 'extension' Definitions

5. 1EdTech Supported QTI Vocabularies & Taxonomies

6. Meta-data Description
     6.1 1EdTech Meta-data
     6.2 QTI-Specific Meta-data

7. Conformance
     7.1 Valid Data Issues
     7.2 Conformance Summary
     7.3 Interoperability Statement

About This Document
     List of Contributors

Revision History

Index

1. Introduction

1.1 Question & Test Interoperability Overview

The Question & Test Interoperability (QTI) specification describes a basic structure for the representation of question (item) and test (assessment) data and their corresponding results reports [QTI, 02i]. Therefore, the specification enables the exchange of this item, assessment and results data between Learning Management Systems, as well as content authors and, content libraries and collections. The QTI specification is defined in XML to promote the widest possible adoption. XML is a powerful, flexible, industry standard markup language used to encode data models for Internet-enabled and distributed applications. The QTI specification is extensible and customizable to permit immediate adoption, even in specialized or proprietary systems. Leading suppliers and consumers of learning products, services and content contributed time and expertise to produce this final specification. The QTI specification, like all 1EdTech specifications, does not limit product designs by specifying user interfaces, pedagogical paradigms, or establishing technology or policies that constrain innovation, interoperability, or reuse.

This document is the 1EdTech QTI Results Reporting Information Model. The first section contains use cases in which the underlying usage, processing control, and core data structures of the results to be reported are described. The basic information model itself is outlined in conceptual terms by using a tabular layout of the context, summary, Assessment results, Section results, and Item results objects in terms of their elements, sub-elements and attributes. The corresponding meta-data, which are used to catalogue these objects, are also described. In addition, the document contains a conformance statement to be used by vendors who plan to implement the specification; we have adopted a descriptive approach to conformance thereby enabling vendors to implement subsets of the full specification.

1.2 Scope & Context

This document is the 1EdTech Question & Test Interoperability: Results Reporting Information Model V1.2 Final Specification document. As such it will be used as the basis for the development of the following documents:

• 1EdTech QTI: Results Reporting XML Binding v1.2 [QTI, 02f];
• 1EdTech QTI: Results Reporting Best Practice & Implementation Guide v1.2 [QTI, 02g].

This requirement has been derived from the agreed 1EdTech Q&TI V1.x Scoping document [QTI, 00]. This Information Model complements the 1EdTech QTI: ASI Information model [QTI, 02a] but it can be applied to assessments that have not been represented using that specification.

1.3 Structure of this Document

The structure of this document is:

2. Specification Use-cases	The definition and scoping of the results that are to be reported using this specification;
3. Basic Information Model	The underlying QTI Results Reporting information model;
4. Conceptual Description of the Data Objects	The detailed description of the QTI Results Reporting data objects in terms of their elements, sub-elements and attributes;
5. 1EdTech Supported QTI Vocabularies & Taxonomies	The definition and description of the vocabularies and taxonomies that are supported as default by 1EdTech;
6. Meta-data Description	The usage of 1EdTech Meta-data and QTI Results Reporting specific meta-data;
7. Conformance	The definition of the conformance statement to be used by vendors.

1.4 Nomenclature

AMS	Assessment Management System
API	Application Programming Interface
ASI	Assessment, Section, Item
CBT	Computer Based Training
DTD	Document Type Definition
LMS	Learning Management System
QTI	Question & Test Interoperability
VLE	Virtual Learning Environment
W3C	World Wide Web Consortium
XML	Extensible Mark-up Language
XSD	XML Schema

1.5 References

[ETS, 99]	A Sample Assessment Using the Four Process Framework, R.Almond, L.Steinberg and R.Mislevy, ETS Working Paper, October 1998.
[1EdTech, 01a]	1EdTech Persistent Location-independent Resource Identifier Implementation Handbook, V1.0, M,McKell, 1EdTech Specification, May, 2001.
[QTI, 00]	1EdTech Question & Test Interoperability Version 1.x Scoping Statement, C.Smythe and E.Shepherd, Version 1.0, 1EdTech, November 2000.
[QTI, 02a]	1EdTech Question & Test Interoperability ASI Information Model Specification, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, 1EdTech, February 2002.
[QTI, 02b]	1EdTech Question & Test Interoperability: ASI XML Binding Specification, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, 1EdTech, February 2002.
[QTI, 02c]	1EdTech Question & Test Interoperability: ASI Best Practice & Implementation Guide, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, 1EdTech, February 2002.
[QTI, 02d]	1EdTech Question & Test Interoperability: ASI Outcomes Processing Specification, C.Smythe, L.Brewer and S.Lay, Final Specification, Version 1.2, 1EdTech, February 2002.
[QTI, 02e]	1EdTech Question & Test Interoperability: ASI Selection & Ordering Specification, C.Smythe, L.Brewer and S.Lay, Final Specification, Version 1.2, 1EdTech, February 2002.
[QTI, 02f]	1EdTech Question & Test Interoperability: Results Reporting XML Binding, C.Smythe, L.Brewer and S.Lay, Final Specification, Version 1.2, 1EdTech, February 2002.
[QTI, 02g]	1EdTech Question & Test Interoperability: Results Reporting Best Practice & Implementation Guide, C.Smythe, L.Brewer and S.Lay, Final Specification, Version 1.2, 1EdTech, February 2002.
[QTI, 02h]	1EdTech Question & Test Interoperability: QTILite Specification, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, 1EdTech, February 2002.
[QTI, 02i]	1EdTech Question & Test Interoperability: Overview, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, 1EdTech, February 2002.

2. Specification Use-Cases

2.1 Basic Architecture

The basic architectural model for the QTI V1.2 specification is shown in Figure 2.1.

 

Figure 2.1 Basic architectural model for QTI V1.2.

In this architecture the ASI repository was introduced in QTI V1.0. This is still supported in QTI V1.2 but the Results repository is now added. The QTI Results Reporting specification is designed to support the following use-cases:

• Permanent storage - the long-term storage of the results in a neutral format;
• Results aggregation - the collation of a set of results from one or more assessment delivery systems and the transfer of those to a results analysis system. This may or may not include the packaging of results from many participants with one or more evaluations per participant. The key point is that the final results may be based upon the collation of date from many different assessment systems;
• Processed results for a Learning Management System (LMS) - the exchange of results between an LMS and the assessment delivery system. This is usually on an individual assessment and a per participant basis. The exchange can also be interactive in that the LMS may respond to the results by launching further assessment and/or content;
• Real-time interactive - the real-time exchange of information concerning the assessment delivery system and the assessment control engine (this may or may not be part of an LMS). The interactive nature means that the delivery system reports the results of a participant's responses etc. which the control engine then processes to determine the suitable reaction e.g. delivery of feedback, delivery of remedial content, etc;
• Internal Interchange Data Language Representation - to act as the internal data representation for a system that consists of a set of communicating processes. This enables portability between different implementations of the processes.

2.2 Results Aggregation System

Assessments may be delivered by various systems such as local area networks, intranets, the Internet or even paper. The results from these assessments need to be collated to accurately compare the results between the various delivery methods. The overall architecture of such a system is shown in Figure 2.2.

 

Figure 2.2 Results aggregation use-case.

In such a system the QTI Results Reporting specification can be used to support the exchange of the information between the 'QTI Results Data' repositories and the 'Results Repository'. These results may be:

• For a single assessment from a single participant - to be contained in a single QTI Results Report structure. The assessment information could be about a single topic, a full high-stake assessment, etc;
• For several assessments for a single participant - to be contained in a single QTI Results Report structure;
• For several assessments from more than one participant - to be contained in multiple QTI Results Report structures and consequently some form of corresponding packaging is required.

2.3 Processed Results for a Learning Management System

A common application is for an LMS to trigger another system to deliver content and collect results. Raw results are not very useful but a score, or accumulation of scores is useful. In this situation an Assessment Management System will determine the scores, and accumulation of scores, and potentially a grade ("A+", "A", "Pass", "Fail", etc). These scores and grades need to be passed back to the LMS. The architecture for such a system is shown in Figure 2.3.

 

Figure 2.3 Processed results for an LMS use-case.

In such a system the QTI Results Reporting specification can be used to support the exchange of the information between the LMS and the AMS. It should be stressed that the QTI specification defines the interface data model for the exchange and not the behavior required on those models. The realization of this interface could involve the appropriate API.

2.4 Real-time Interactive

Certain systems in the marketplace require the raw answers from an item delivery system to allow it to analyze and provide a score, descriptive and/or prescriptive feedback. For example an essay grading system (results analysis system) might accept an essay from an item delivery system, grade the essay, generate feedback and pass this back to the item delivery system. Another example is several items are delivered by the item delivery system, on various topics, and the results analysis system generates specific topic scores and feedback. The architecture for such a system is shown in Figure 2.4.

 

Figure 2.4 Raw answers for a results analysis use-case.

In such a system the 1EdTech QTI Results Reporting specification can be used to support the exchange of the information between the 'Results Analysis System' and the 'Item Delivery System'. It should be stressed that the QTI specification defines the interface data model for the exchange and not the behavior required on those models. The realization of this interface could involve the appropriate API.

2.5 Internal Interchange Data Language Representation

In some cases the QTI specification may be adopted to support the internal representation of the results information. This representation is then used for the exchange of data between processes within a single system. An example of such an approach is shown in Figure 2.5 - this is based upon the four-process model created by ETS [ETS, 99].

 

Figure 2.5 Internal data language representation use-case.

In such a system the QTI Results Reporting specification can be used to support the exchange of the information between the 'Item Response Processing' and Assessment/section Processing' processes. It could also be used for sub-process communication within each of these macro processes.

3. Basic Information Model

The underlying logical data structures for the QTI Results Reporting package are shown in Figure 3.1. Figure 3.1 shows the four core structures that can be exchanged using the QTI Results Reporting specification.

 

Figure 3.1 The principal QTI results reporting data structures.

The core data structures are:

• result - the set of results relevant to an actual attempt of an assessment or some other form of evaluation. Multiple results for a single or multiple participants can be contained within the QTI Results Reporting package;
• context - the contextual information concerning the actual evaluation being reported e.g. the name of the participant, participant identifiers, etc;
• summary_result- the summary information for a particular instance of the evaluation. Each result can contain only one set of summary information;
• assessment_result - the detailed assessment information for a particular attempt at the assessment. Each result can contain information about one assessment only (including any contained sections and items);
• section_result - the detailed information about the section(s) completed, or to being attempted. Each result can contain information about one section (including any contained sections and/or items);
• item_result - the detailed information about the item(s) completed, or to being attempted. Each result can contain information about one item.

This structure has been adopted so that:

• Both summary or detailed results information can be exchanged depending on which is required. The aim is to minimize the overheads required to capture the key data. This means that the different use-cases are supported in an optimal manner;
• The relationship between the 1EdTech QTI ASI information model is immediately obvious. The Results Reporting specification can be used to support assessments etc. that have not be based upon the 1EdTech QTI ASI model however this will require some form of mapping to the QTI representational model and/or the use of the appropriate extensions;
• As a compromise between a representation that is neither too abstract nor too implementation specific. The underlying model supports extension mechanisms in a several areas.

4. Conceptual Description of the Data Objects

4.1 Underlying Structure of the Results Report Package

The primary elements of the learner information are shown in Figure 4.1:

 

Figure 4.1 The primary elements of the QTI results reporting data structures.

4.2 Extensions & Extensibility

A key requirement for the specification is its support, where appropriate, for extensions. These extensions take two forms:

• Vocabulary extensions - extensions that allow the basic vocabularies to be extended. These vocabularies are assigned to many of the data objects and they are used to define the type of information being contained. The basic vocabularies are defined within a set of default files maintained by 1EdTech. The vocabulary extensions may be included in a string list or referenced using a URI;
• Functional extensions - extensions that are included to ensure that users of the specification can add functionality that is otherwise excluded from the specification (in the following tabular descriptions these are denoted by the 'extension' data structure). Further versions of the specification make full commitment to ensure backwards compatibility with these features. Within the XML binding each of these extensions will be given a unique element name.

The process by which the two forms of extension fit within the taxonomy is clearly denoted in the 1EdTech QTI Results Reporting XML Binding [QTI, 02f].

4.3 QTI Results Report Package Tabular Description

The tables in this Section provide a conceptual, informative description of the elements in the data objects. The columns in these tables refer to:

No:	The number of the data element. An element may be composed of sub-elements. The numbering scheme reflects these relationships.
Name:	The descriptive name of the element.
Explanation:	A brief functional description of the element.
Required:	Indicates if the element is required: M = Mandatory Element that must be included in the data object, if the element at the higher level is included; C = Conditional Element. Existence is dependent on values of other Elements; O = Optional Element.
Multi:	Multiplicity of the element: Blank = single instance; Number = maximum number of times the element is repeatable; n = multiple occurrences allowed, no limit; Repeatability of an element implies that all sub-elements repeat with the same element.
Type:	A description of formatting rules for the data element. Type includes the maximum length of the element: ID = element used to uniquely identify an object; Code = element value from a list of codes; Description = descriptive element, human language Flag = binary flag Enumerated = list of predefined non-numeric options i.e. the definitive list of objects The international character set specified by ISO 10646 will be used for all fields. The type will also include a description of the set of valid values for the sub-element: Coding schemes using numerical values; The set of values as defined in the Domain i.e. making it closed. The list of values cannot be extended to include values not defined in the specification. If there is a need for values not included in the domain set of values then the extension should be done defining a new element under the Extension element that is a part of each data object definition.

Note: Additional descriptive information about the element.

In the following tables the data objects are organized as:

• Table 4.1 - the QTI Results Reporting data structure i.e. the structure that constitutes the report package itself;
• Table 4.2 - the 'summary_result' data structure;
• Table 4.3 - the 'assessment_result' data structure;
• Table 4.4 - the 'section_result' data structure;
• Table 4.5 - the 'item_result' data structure;
• Table 4.6 - the common data structures (these are used within more than one of the above data structures);
• Table 4.7 - the functional extensions, <extension>, that are supported.

4.3.1 QTI Results Reporting Data Objects

Table 4.1 describes the data objects that are used in the construction of the QTI results reporting package itself.

 

Table 4.1 QTI results reporting data objects detailed description.

No	Name	Explanation	Reqd	Mult	Type	Note
1.1	qti_comment	A comment about the full QTI results report structure. As per structure 6.1 (Table 4.6).
1.2	result	The container for the a set of results for a particular participant in a particular context	M	n		Reports of multiple evaluations use multiple result instances.
1.2.1	qti_comment	A comment about the context of the evaluation. As per structure 6.1 (Table 4.6).
1.2.2	context	Contains the information about the participant that sets the context for the evaluation being reported.	M			A context is mapped on a one-to-one relationship with an evaluation.
1.2.2.1	qti_comment	A comment about the context of the evaluation. As per structure 6.1 (Table 4.6).
1.2.2.2	name	The detailed formatted name of the participant.	O		#PCDATA String. 1-256 chars.
1.2.2.3	generic_identifier	The set of identifiers that are used to define the participant and their evaluation environment. Each identifier is vocabulary based.	O	n	As per structure 6.4 (Table 4.6).
1.2.2.4	date	The set of dates that are relevant to the context of the evaluation e.g. the creation date of the report. Each date is vocabulary based.	O	n	As per structure 6.5 (Table 4.6).
1.2.2.5	extension	The extension facility for the 'context' data structure.	O		As per structure 7.1 (Table 4.7).
1.2.3	summary_result	The set of summary information concerning a particular assessment completed by a participant.	C		As per Table 4.2.
1.2.4	assessment_result	The set of detailed results information for a particular assessment. An assessment is as defined within the QTI specifications.			As per Table 4.3.
1.2.5	sectionresult	The set of detailed results information for the section. A section is as defined within the QTI specifications.			As per Table 4.4.
1.2.6	item_result	The set of detailed results information for the item. An assessment is as defined within the QTI specifications.			As per Table 4.5.
1.2.7	extension	The extension facility for the 'instance' data structure.			As per structure 7.2 (Table 4.7).

4.3.2 'summary_result' Data Objects

Table 4.1 describes the data objects that are used in the construction of the summary results information.

 

Table 4.2 'summary_result' data objects detailed description.

No	Name	Explanation	Reqd	Mult	Type	Note
2.1	asi_title	The title of the evaluation being reported. This would be identical to that assigned under the QTI ASI spec.	O		As per structure 6.6 (Table 4.6).
2.2	qti_comment	As per structure 6.1 (Table 4.6).
2.3	type_label	The type of summary results being reported e.g. an assessment, item, etc.			As per structure 6.2 (Table 4.6).
2.4	generic_identifier	The generic identifiers that are allocated to different aspects of the summary information.	O	n	As per structure 6.4 (Table 4.6).
2.5	date	The dates that are relevant to the summary e.g. the date of creation.	O	n	As per structure 6.5 (Table 4.6).
2.6	status	The status of the evaluation e.g. completed, in-progress, etc.	O		As per structure 6.11 (Table 4.6).
2.7	duration	The time taken for the completion of the evaluation.	O		As per structure 6.12 (Table 4.6).
2.8	score	The score assigned to the evaluation using the default variable i.e. 'SCORE', 'CORRECT' or 'GRADE'.	O		As per structure 6.13 (Table 4.6).
2.9	grade	The grade that has been awarded to the participant e.g. 'Pass', 'Fail', etc.	O		As per structure 6.14 (Table 4.6).
2.10	outcomes	The data structure that contains the extended set of outcome results.	O	n	As per structure 6.15 (Table 4.6).
2.11	extension	The extension facility to the <summary> data structure.	O		As per structure 7.3 (Table 4.7).

4.3.3 'assessment_result' Data Objects

Table 4.3 describes the data objects that are used in the construction of the detailed assessment results.

 

Table 4.3 'assessment_result' data objects detailed description.

No	Name	Explanation	Reqd	Mult	Type	Note
3.1	asi_title	The title of the assessment being reported. This would be identical to that assigned under the QTI ASI spec.	O		As per structure 6.6 (Table 4.6).
3.2	ident_ref	The formal identifier associated with this assessment. This would be identical to that assigned under the QTI ASI spec.	O		As per structure 6.7 (Table 4.6).
3.3	qti_comment	As per structure 6.1 (Table 4.6).
3.4	asi_metadata	The QTI- results reporting specific meta-data that is associated with this assessment.	O	n	As per structure 6.8 (Table 4.6).
3.5	asi_description	A brief description of the Assessment. The description can be contained in an external file.	O		As per structure 6.22 (Table 4.6).
3.6	date	The dates that are relevant to the assessment result record e.g. the date of creation.	O	n	As per structure 6.5 (Table 4.6).
3.7	duration	The time taken for the completion of the assessment.	O		As per structure 6.12 (Table 4.6).
3.8	objective	The set of objectives that are covered by this assessment for the type of participant(s).	O	n	As per structure 6.9 (Table 4.6).
3.9	control	The feedback mechanisms that were revealed to the participant(s)	O		As per structure 6.10 (Table 4.6).
3.10	outcomes	The data structure that contains all of the scores being reported as part of the assessment.	O		As per structure 6.15 (Table 4.6).
3.11	feedback_displayed	An indication of the type of feedback that was given to the participant	O	n	As per structure 6.21 (Table 4.6).
3.12	num_items	The accumulated number of Items that are related to this assessment	O		As per structure 6.16 (Table 4.6).
3.13	num_sections	The number of Sections that are directly related to the parent assessment	O		As per structure 6.17 (Table 4.6).
3.14	num_items_presented	The number of items presented to the participant during the assessment.	O		As per structure 6.18 (Table 4.6).
3.15	num_items_attempted	The number of items attempted by the participant during the assessment.	O		As per structure 6.19 (Table 4.6).
3.16	num_sections_presented	The number of sections presented to the participant during the assessment.	O		As per structure 6.20 (Table 4.6).
3.17	section_result	The set of detailed results information for each of the prescribed sections. A section is as defined within the QTI specifications.	O	n	As per Table 4.3.
3.18	extension	The extension facility for the assessment results.	O		As per structure 7.4 (Table 4.7).

4.3.4 'section_result' Data Objects

Table 4.4 describes the data objects that are used in the construction of the detailed section results.

 

Table 4.4 'section_result' data objects detailed description.

No	Name	Explanation	Reqd	Mult	Type	Note
4.1	asi_title	The title of the assessment being reported. This would be identical to that assigned under the QTI ASI specification.	O		As per structure 6.6 (Table 4.6).
4.2	ident_ref	The formal identifier associated with this section. This would be identical to that assigned under the QTI ASI specification.	O		As per structure 6.7 (Table 4.6).
4.3	presented	This is used to indicate whether or not the associated Section was presented to the participant.	O		Enumerated: Yes (default) No Unknown	This is required because the full structure may consist of nested Sections only some of which may be presented i.e. due any associated selection & ordering algorithm.
4.4	qti_comment	As per structure 6.1 (Table 4.6).
4.5	asi_metadata	The QTI- results reporting specific meta-data that is associated with this section.	O	n	As per structure 6.8 (Table 4.6).
4.6	asi_description	A brief description of the Section. The description can be contained in an external file.	O		As per structure 6.22 (Table 4.6).
4.7	date	The dates that are relevant to the section result record e.g. the date of creation.	O	n	As per structure 6.5 (Table 4.6).
4.8	duration	The time taken for the completion of the section.	O		As per structure 6.12 (Table 4.6).
4.9	objective	The set of objectives that are covered by this section for the type of participant(s).	O	n	As per structure 6.9 (Table 4.6).
4.10	control	The feedback mechanisms that were revealed to the participant(s)	O		As per structure 6.10 (Table 4.6).
4.11	outcomes	The data structure that contains all of the scores being reported as part of the section.	O		As per structure 6.15 (Table 4.6).
4.12	feedback_displayed	An indication of the type of feedback that was given to the participant	O	n	As per structure 6.21 (Table 4.6).
4.13	num_items	The accumulated number of Items that are related to this assessment	O		As per structure 6.16 (Table 4.6).
4.14	num_sections	The number of Sections that are directly related to the parent assessment	O		As per structure 6.17 (Table 4.6).
4.15	num_items_presented	The number of items presented to the participant during the section evaluation.	O		As per structure 6.18 (Table 4.6).
4.16	num_items_attempted	The number of items attempted by the participant during the section.	O		As per structure 6.19 (Table 4.6).
4.17	num_sections_presented	The number of sections presented to the participant during the assessment.	O		As per structure 6.20 (Table 4.6).
4.18	section_result	The set of detailed results information for each of the prescribed sections. A section is as defined within the QTI specifications.	O	n	As per Table 4.3.
4.19	item_result	The set of detailed results information for each of the prescribed items. An Item is as defined within the QTI specifications.			As per Table 4.4.
4.20	extension	The extension facility for the section results.	O		As per structure 7.5 (Table 4.7).

4.3.5 'item_result' Data Objects

Table 4.5 describes the data objects that are used in the construction of the QTI results reporting package itself.

 

Table 4.5 'item_result' data objects detailed description.

No	Name	Explanation	Reqd	Mult	Type	Note
5.1	asi_title	The title of the item being reported. This would be identical to that assigned under the QTI ASI specification.	O		As per structure 6.6 (Table 4.6).
5.2	ident_ref	The formal identifier associated with this item. This would be identical to that assigned under the QTI ASI specification.	O		As per structure 6.7 (Table 4.6).
5.3	presented	This is used to indicate whether or not the associated Item was presented to the participant.	O		Enumerated: Yes (default) No Unknown	This is required because the full structure may consist of many Items only some of which may be presented i.e. due any associated selection & ordering algorithm.
5.3	qti_comment	As per structure 6.1 (Table 4.6).
5.4	asi_metadata	The QTI- results reporting specific meta-data that is associated with this item.	O	n	As per structure 6.8 (Table 4.6).
5.5	asi_description	A brief description of the Item. The description can be contained in an external file.	O		As per structure 6.22 (Table 4.6).
5.6	date	The dates that are relevant to the item result record e.g. the date of creation.	O	n	As per structure 6.5 (Table 4.6).
5.7	duration	The time taken for the completion of the section.	O		As per structure 6.12 (Table 4.6).
5.8	objective	The set of objectives that are covered by this item for the type of participant(s).	O	n	As per structure 6.9 (Table 4.6).
5.9	control	The feedback mechanisms that were revealed to the participant(s)	O		As per structure 6.10 (Table 4.6).
5.10	response	The response from the participant for a particular Item response block.	M	n
5.10.1	ident_ref	The identifier for the response block as assigned by the 'ident' attribute under the QTI ASI specification.	O		As per structure 6.7 (Table 4.6).
5.10.2	qti_comment	As per structure 6.1 (Table 4.6).
5.10.3	responseform	The information that describes the form of the response structure being presented.	O
5.10.3.1	response_type	The type of response that is expected from the participant.	O		Enumerated list from: 'lid' 'xy', 'str', 'num', 'grp' 'extension'
5.10.3.2	cardinality	The cardinality of the response block.	O		Enumerated list from: 'single' 'multiple' 'ordered' 'extension'
5.10.3.3	render_type	The type of rendering mechanism used.	O		Enumerated list: 'choice' 'hotspot' 'slider' 'fib' 'extension'
5.10.3.4	timing	The time-based dependence of the response.	O		Enumerated list: 'Yes' 'No'
5.10.3.5	correct_response	The correct answer for this response. This will take the form of the label for the identifier or the actual answer itself.	O	n	#PCDATA String. 1-256 chars.
5.10.3.6	extension	The extension facility for the description of the response structure.	O		As per structure 7.7 (Table 4.7).
5.10.4	num_attempts	The number of attempts made to answer the Item.	O		#PCDATA String 1-2 chars.	The value '0' means that the question was not attempted.
5.10.5	response_value	The sequenced responses given by the participant.	O	n	#PCDATA String. 1-256 chars.	The order and number of responses matches those given by the participant. This information should match with the cardinality of the original Item.
5.10.5.1	response_time	The time in seconds between the start of the evaluation event e.g. the start of the Item, and the entry of the response.	O		#PCDATA String. 1-5 chars
5.10.5.2	response_status	The status of the attempt to answer this Item.	O		Enumerated list: 'Null' 'Valid' 'NA' 'Invalid'. 'Valid' is the default value.	If a valid attempt has been made to answer the Item then the status is 'Valid'. If no attempt was made then the status is 'NA'. If the no information is available then the status is 'Null'. An 'Invalid' status is returned to indicate a system failure condition.
5.10.5.3	uri	The location of the object.	C		CDATA String 256 As per RFC1630.	An alternative to this is the 'entityref' attribute cf. 5.10.5.4).
5.10.5.4	entityref	The entity reference that is used to bind an external file to the XML instance.			CDATA string defining the link. String 256.	This is an alternative to the 'uri' attribute (cf. 5.10.5.3).
5.10.5	extension	The extension facility for the participant responses.	O		As per structure 7.8 (Table 4.7).
5.11	outcomes	The data structure of the scores being reported as part of the assessment.	O		As per structure 6.15 (Table 4.6).
5.12	feedback_displayed	The type of feedback that was presented to the participant.	O	n	As per structure 6.21 (Table 4.6).
5.13	extension	The extension facility for the item results.	O		As per structure 7.6 (Table 4.7).

4.3.6 Common Data Objects

Table 4.6 describes the common data objects that are used to support the other data objects.

 

Table 4.6 Common data objects detailed description.

No	Name	Explanation	Reqd	Mult	Type	Note
6.1	qti_comment	Comments of the QTI Results Reporting data objects. These comments are not removed by an XML parser.	O		#PCDATA String 1-2048 chars.	These comments should be used to annotate the XML instances.
6.1.1	language_type	The language that is being used for the information.	As per structure 6.3.
6.2	type_label	The label of the object that is being defined.	O			The naming convention should reflect the usage of the name. See Section 5 for a further explanation of the vocabularies that must be supported.
6.2.1	source	The source of the vocabulary.	O		String 1-256 chars.	Contains the name of the external file containing vocabulary.
6.3	language_type	The language that is being used for the information. This will be realized using 'xml:lang'.	O		String 1-256 chars.	The language entries will be defined as per the ISO639 standard and RFC1766.
6.4	generic_identifier	The generic identifier.	O	n		It is expected that this object will conform to the 1EdTech Persistent Location-independent Resource Identifier handbook.
6.4.1	type_label	The type of generic identifier as defined by the associated vocabulary.	O		As per structure 6.2.
6.4.2	identifier_string	A unique identifier for the data record assigned by the creating entity.	M		#PCDATA String 1-256 chars.	The unique identifier assigned by reference to the object. Can contain any character.
6.5	date	The date and/or time to be recorded.	O	n
6.5.1	type_label	The type of date as defined by the associated vocabulary.	O		As per structure 6.2.
6.5.2	datetime	The underlying string structure is: YYYY:MM:DDT HH:MM:SS	M		#PCDATA String. 1-20 chars.	Defined as per ISO 8601.
6.6	asi_title	The title of the evaluation being reported. This would be identical to that assigned under the QTI ASI specification.	O		#PCDATA String. 1-256 chars.
6.7	ident_ref	The formal identifier associated with this evaluation. This would be identical to that assigned under the QTI ASI specification.	O		#PCDATA String. 1-256 chars.	This should conform to the 1EdTech Persistent Location-independent Resource Identifier specification. Can contain any character.
6.8	asi_metadata	The container for all of the vocabulary-based QTI-specific meta-data.	O			Multiple vocabularies can be used for each set of meta-data fields.
6.8.1	vocabulary	The vocabulary to be applied to the associated meta-data fields.	O		#PCDATA String 2048 chars.	If no file is used then the vocabulary can be contained as a comma delimited string.
6.8.1.1	uri	The location of the object.	C		CDATA String 256 chars. As per RFC1630.	An alternative to this is the 'entityref' attribute cf. 6.8.1.2).
6.8.1.2	entityref	The entity reference that is used to bind an external file to the XML instance.			CDATA string defining the link. String 256 chars.	This is an alternative to the 'uri' attribute (cf. 6.8.1.1)
6.8.1.3	vocab_type	This is the type of vocabulary file that is identified in either the associated 'uri' or 'entityref' attributes.	O		CDATA string type of file. String 16 chars.
6.8.2	asi_metadatafield	The QTI- results reporting specific meta-data that is associated with this evaluation.	M	n
6.8.2.1	language_type	The language that is being used for the meta-data content.	As per structure 6.3.
6.8.2.2	field_name	The field label as taken from the possible vocabulary.	M		#PCDATA String 256 chars.
6.8.2.3	field_value	The data for that field.	M		#PCDATA String 256 chars.	Its format will generally be either a text string or numerical.
6.9	objective	The set of objectives that are covered by this evaluation for the type of participant(s).	O
6.9.1	view	The view to which the interpretation is applied.	O	n	Enumerated: All (default) Administrator AdminAuthority Assessor Author Candidate Invigilator Proctor Psychometrician Scorer Tutor	The 'All' view is the default value.
6.9.2	uri	The location of the objectives file.	C		CDATA String 256 chars. As per RFC1630.	An alternative to this is the 'entityref' attribute cf. 6.9.3).
6.9.3	entityref	The entity reference that is used to bind an external file to the XML instance.			CDATA string defining the link. String 256 chars.	This is an alternative to the 'uri' attribute (cf. 6.9.2)
6.10	control	The feedback mechanisms that were revealed to the participant(s)	O		#PCDATA String 128 chars.	This will be a string that contains information describing the particular feedback given.
6.10.1	hint_switch	Used to show if hints had been displayed to the participant.	O		Enumerated: Yes No (default)
6.10.2	solution_switch	Used to show if solutions had been displayed to the participant.	O		Enumerated: Yes No (default)
6.10.3	feedback_switch	Used to show if feedback had been displayed to the participant.	O		Enumerated: Yes No (default)
6.11	status	The status of the evaluation that is/has been undertaken.	O
6.11.1	type_label	The vocabulary for the field.	O		As per structure 6.2.
6.11.2	datetime	The date at which the status is recorded.	O		As per structure 6.5.2.
6.11.3	status_value	The status of the event.	M		#PCDATA String. 1-64 chars.
6.12	duration	The time taken for the completion of the section.	O		#PCDATA String. 1-32 chars.	Defined as per ISO 8601. The underlying string structure is: PnYnMnDT nH:nM:nS
6.13	score	The score and associated descriptive information for the evaluation.	O
6.13.1	varname	The variable name being used to contain the score.	O		#PCDATA String. 1-256 chars.
6.13.2	vartype	The type of scoring variable.	O		Enumerated: String Integer (Default) Decimal Scientific Boolean Set
6.13.3	status	The status of the score being described.	O		Enumerated: Valid (default) Noanswer Error	The error value is assigned if an error was generated when undertaking the response processing or outcomes processing.
6.13.4	score_value	The actual score value being reported.	O		#PCDATA String. 1-32 chars.
6.13.5	score_interpretation	A description of the way in which the scores should be interpreted.	O		#PCDATA String. 1-2048 chars.
6.13.5.1	uri	The location of the interpretation file.	C		CDATA String 256 chars. As per RFC1630.	An alternative to this is the 'entityref' attribute cf. 6.13.5.2).
6.13.5.2	entityref	The entity reference that is used to bind an external file to the XML instance.			CDATA string defining the link. String 256 chars.	This is an alternative to the 'uri' attribute (cf. 6.13.5.1)
6.13.6	score_min	The minimum score that could have been awarded to the evaluation.	O		#PCDATA String real number. 32 chars	This is only valid for numeric variables.
6.13.7	score_max	The maximum score that could have been awarded to the evaluation.	O		#PCDATA String real number. 32 chars	This is only valid for numeric variables.
6.13.8	score_normalized	The normalized value for the score i.e. a number in the range of 0-1.	O		#PCDATA String real number. 32 chars	The number is normalized using the minimum and maximum values. This is only valid for numeric variables.
6.13.9	score_average	The average score that would be expected when undertaking the evaluation.	O		#PCDATA String real number. 32 chars	This information is not derived from the data-set itself. This is only valid for numeric variables.
6.13.10	score_std_error	The standard error associated with the score.	O		#PCDATA String real number. 32 chars	This information is not derived from the data-set itself. This is only valid for numeric variables.
6.13.11	score_reliability	A numerical indication of the reliability of the score.	O		#PCDATA String real number. 32 chars	This information is not derived from the data-set itself.
6.13.12	score_cut	The cut score which when achieved or surpassed denotes mastery of the topic.	O		#PCDATA String 32 chars
6.13.13	extension	The extension facility for the scores.	O		As per structure 7.9 (Table 4.7).
6.14	grade	The grading information assigned to the result of this evaluation.	O
6.14.1	varname	The variable name being used to contain the grade.	O		#PCDATA String. 1-256 chars.
6.14.2	members	The possible set of grades that could have been awarded.	O		#PCDATA String 1-2048.	This will be a comma separated list.
6.14.3	status	The status of the grade being described.	O		Enumerated: Valid (default) Noanswer Error	The error value is assigned if an error was generated when undertaking the response processing or outcomes processing.
6.14.4	grade_value	The grade that has been awarded.	O		#PCDATA String 1-64.
6.14.5	grade_cut	The cut grade which when achieved or exceeded denotes mastery of the topic.	O		#PCDATA String 1-64.
6.14.6	extension	The extension facility for the grades.	O		As per structure 7.10 (Table 4.7).
6.15	outcomes	The set of scores and/or grades that are to be reported for the evaluation.	O
6.15.1	status	The status of the report.	O		As per structure 6.11.
6.15.2	score	The details of a score assigned to the evaluation.	O	n	As per structure 6.13.
6.15.3	grade	The details of the grade assigned to the evaluation.	O	n	As per structure 6.14.
6.16	num_items	The number of accumulated Items that are contained in the evaluation.	O		#PCDATA String 1-32 chars.
6.17	num_sections	The number of Sections that are directly contained in the parent.	O		#PCDATA String 1-32 chars.
6.18	num_items_presented	The number of items that have been presented to the participant as part of the evaluation.	O		#PCDATA String 1-32 chars.
6.19	num_items_attempted	The number of items that have been attempted by the participant during the evaluation.	O		#PCDATA String 1-32 chars.
6.20	num_sections_presented	The number of sections that have been presented to the participant as part of the evaluation.	O		#PCDATA String 1-32 chars.
6.21	feedback_displayed	The type of feedback that was presented to the participant.	O	n	#PCDATA String. 1-10 chars.	This will take one of the values: 'Response', 'Hint' or 'Solution'.
6.21.1	ident_ref	The formal identifier associated with this feedback. This would be identical to that assigned under the QTI ASI specification.	O		As per structure 6.7 (Table 4.6).
6.21.2	asi_title	The title of the feedback being reported. This would be identical to that assigned under the QTI ASI specification.	O		As per structure 6.6 (Table 4.6).
6.21.3	uri	The location of the feedback file.	C		CDATA String 256 chars. As per RFC1630.	An alternative to this is the 'entityref' attribute cf. 6.20.4).
6.21.4	entityref	The entity reference that is used to bind an external file to the XML instance.			CDATA string defining the link. String 256 chars.	This is an alternative to the 'uri' attribute (cf. 6.20.3)
6.22	asi_description	A description of the form of the evaluation object that was presented to the participant.	O		#PCDATA String. 1-2048 chars.
6.22.1	uri	The location of the description file.	C		CDATA String 256 chars. As per RFC1630.	An alternative to this is the 'entityref' attribute cf. 6.21.2).
6.22.2	entityref	The entity reference that is used to bind an external file to the XML instance.			CDATA string defining the link. String 256 chars.	This is an alternative to the 'uri' attribute (cf. 6.21.1)

4.3.7 'extension' Definitions

Table 4.7 list the set of extension names that are used within the XML binding. These names are mapped to the 'extension' element used within Tables 4.1 to 4.6 of the Information Model.

 

Table 4.7 Common data objects detailed description.

No	Name	Source Element	Usage
7.1	extension_context	context	Extension for the 'Context' data structure.
7.2	extension _result	result	Extension for the 'Result' core data structure.
7.3	extension _summary_result	summary	Extension for the 'Summary' core data structure.
7.4	extension _assessment_result	assessment_result	Extension for the 'Assessment' core data structure.
7.5	extension _section_result	section_result	Extension for the 'Section' core data structure.
7.6	extension_item_result	item_result	Extension for the 'Item' core data structure.
7.7	extension _responseform	responseform	Extension for the 'Responseform' data structure. Supports any proprietary features that describe the response structure presented to the participant.
7.8	extension_response	response	Extension for the 'Response' data structure. Supports any proprietary features used in the original response block structure.
7.9	extension _score	score	Extension for the scoring information for a particular variable.
7.10	extension _grade	grade	Extension for the grading information.

5. 1EdTech Supported QTI Vocabularies & Taxonomies

Within the QTI Results Reports there are many special vocabularies that are required to define the specific type of information being included. These vocabularies and their default 1EdTech file name are listed in Table 5.1.

 

Table 5.1 'typename' list of vocabularies.

No	Source Element	File Name	Default Vocabulary
1	generic_identifier	imsqtiv1p2_identifier.txt	TBD.
2	date	imsqtiv1p2_date.txt	TBD.
3	status	imsqtiv1p2_status.txt	TBD.
4	summary_result	imsqtiv1p2_summary.txt	TBD.
5	asi_metadata	imsqtiv1p2_metadata.txt	See sub-section 6.2.

A request is made to those organizations adopting the 1EdTech QTI results Reporting specification to inform the 1EdTech QTI working-group of the vocabulary entries that they require.

6. Meta-data Description

6.1 1EdTech Meta-data

Support for the inclusion of the 1EdTech Meta-data is through the usage of the 1EdTech Content Packaging mechanism.

6.2 QTI-Specific Meta-data

The vocabulary for the QTI-specific meta-data is defined in Table 6.1. The vocabulary is supplied as a comma separated list in the text file "imsqtiv1p2_metadata.txt". In Table 6.1 and entry is either optional ('O') or not applicable ('N/A') for the results of a particular evaluation object (Assessment, section or Item result).

 

Table 6.1 Assessment, Section and Item meta-data list comparison.

Field	1EdTech Class	Assess Result Class	Section Result Class	Item Result Class
qmd_absolutescore_max	The maximum score that the user may attain (Real number: 1-9999).	O	O	O
qmd_absolutescore_min	The minimum score that the user may attain (Real number: 1-9999).	O	O	O
qmd_assessmenttype	The type of assessment role: Examination Survey Tutorial Self-assessment.	O	N/A	N/A
qmd_computerscored	Whether or not the item can be computer scored [Yes/No].	N/A	N/A	O
qmd_feedbackapermitted	Whether or not the feedback is to be made available. Value: [Yes/No] with default=Yes.	O	O	O
qmd_hintspermitted	Whether or not the hints are to be made available [Yes/No] with default=Yes.	O	O	O
qmd_itemselection	Whether or not Section sequencing is available. Value: [Yes/No] with default=Yes.	O	O	N/A
qmd_itemsequence	Whether or not Item selection is available. Value: [Yes/No] with default=Yes.	O	O	N/A
qmd_itemtype	The type of Item (list including): Logical identifier XY co-ordinate String Numeric Logical group Composite Proprietary	N/A	N/A	O
qmd_levelofdifficulty	The level of difficulty of the Item (list including): Pre-school School Higher/further education Vocational Professional Development	N/A	N/A	O
qmd_material	Listing of the types of content supplied in the Item: Text/basic Text/rtf Text/html Text/xhtml Image/gif Image/jpeg Audio/aicc Audio/wav Video/quicktime 3 Video/quicktime 4 Video/avi Video/mpeg 1 Video/mpeg2 Video/mpeg4 Applet/java Application	O	O	O
qmd_numberofitems	The number of Items directly referenced within the Section.	N/A	O	N/A
qmd_penaltyvalue	The penalty value that is to be used with the GuessPenalty scoring algorithms. This is a number in the range -231-3 to 231-1.	O	O	O
qmd_questiontype	The type of question (list including): True/false Multiple-choice Multiple-response FIB-string FIB-numeric Image hot-spot Drag-and-drop Essay	N/A	N/A	O
qmd_renderingtype	The type of rendering employed (list including): Choice Hot spot Slider Text entry Proprietary	N/A	N/A	O
qmd_responsetype	The class of responses required by the Item: Single Multiple Ordered Proprietary	N/A	N/A	O
qmd_scorereliability	The reliability metric that has been assigned to the score of this evaluation object.	O	O	O
qmd_scorestderr	The standard error that has been assigned to the score for this evaluation object.	O	O	O
qmd_scoretype	The scoring classification: Absolute Percentage Unscored Multidimensional	O	O	N/A
qmd_scoringpermitted	Whether or not scoring is enabled. Value: [Yes/No] with default=Yes.	N/A	N/A	O
qmd_sectionselection	Whether or not the solutions are available. Value: [Yes/No] with default=Yes.	O	O	N/A
qmd_sectionsequence	Whether or not Section selection is available. Value: [Yes/No] with default=Yes.	O	O	N/A
qmd_sectionsincluded	Whether or not other Sections are defined within the Section [Yes/No].	N/A	O	N/A
qmd_solutionspermitted	Whether or not the solutions are to be made available. Value: [Yes/No] with default=Yes.	O	O	O
qmd_status	The status of the Item: Experimental Normal Retired.	N/A	N/A	O
qmd_timedependence	Whether or not the user responses are time dependent. Value: [Yes/No] with default=No.	N/A	N/A	O
qmd_timelimit	The number of minutes or an unlimited duration.	O	O	O
qmd_toolvendor	Name of the vendor of the tool creating the Assessments.	O	N/A	O
qmd_topic	A brief description of the topic covered by the Item.	N/A	O	O
qmd_typeofsolution	The type of solution supplied by the Item: Complete Incremental Multilevel Proprietary	N/A	N/A	O
qmd_versionnumber	The version number that has been assigned to the object. The nomenclature for the versioning is outside the scope of this specification. Value: String 1-256 chars.	O	O	O
qmd_weighting	The weighting of the Item to be applied in scoring. Real number: 0.01-10	N/A	O	O

The context for the 1EdTech QTI-specific meta-data is taken from the object in which the meta-data is declared (the listed meta-data would refer to the results of the identified 'Item'):

  <qti_result_report>
    <context>
      ...
    </context>
    <item_result ident_ref="***...***">
      ...
      <asi_metadata>
        <vocabulary uri="imsqtiv1p2_metadata.txt" vocab_type="text/plain"/>
        <asi_metadatafield>
          <field_name>qmd_questiontype</field_name>
          <field_value>Multiple-choice</field_value>
        </asi_metadatafield>
        ...
      </asi_metadata>
      ...
    </item_result>
  </qti_result_report>

7. Conformance

The purpose of this statement is to provide a mechanism for customers to fairly compare vendors of assessment systems, tools and content. It is not mandatory for a vendor to support every feature of the QTI Results Reporting specification, but a vendor must detail their level of support with a "Conformance Statement". For example vendors may choose to accept or publish QTI data, but not choose to repackage QTI data. Compliance is represented by:

• Conformance summary - this is a summary that shows, in colloquial terms, the capabilities of a particular implementation with respect to the 1EdTech QTI Results Reporting specification;
• Interoperability statement - this is a detailed technical checklist that identifies all of the feature capabilities of the implementation in terms of the 1EdTech QTI Results Reporting specification functions.

7.1 Valid Data Issues

Vendors claiming conformance shall publish, accept, and/or repackage valid 1EdTech QTI Results Reporting data as defined by the DTD including proprietary extensions where applicable. Vendors claiming their tools publish QTI Results Reports shall export valid QTI data. Vendors claiming their system tools accept QTI Results Reports data shall be able to parse and recognize valid QTI data. Vendors claiming their system tools repackage QTI Results Reports data shall be able "pass through" valid QTI Results Reports data whether the tool recognizes the optional elements or not. Vendors claiming their assessment content conforms to this specification shall provide valid QTI Results Reports data. Publishers claiming their content conforms to QTI Results Reports shall provide valid QTI Results Reports data.

7.2 Conformance Summary

Vendors claiming conformance must provide a "Conformance Summary", detailing their level of conformance, substantially similar to the information shown below, upon a reasonable request from a member of the 1EdTech, or a prospective customer(s). It is expected that this table, a template of which is shown in Table 7.1, is a summary of the information given in the 'Interoperability statement'. The intention is for the 'Conformance Summary' to be informative in nature.

 

Table 7.1 QTI Results Report conformance summary.

	QTI Results Report Conformance Summary (Version 1.2)
	Publish (export, data)	Accept (import, display)	Repackage Feature
context	Y or N	Y or N	Y or N
summary result	Y or N	Y or N	Y or N
score	Y or N	Y or N	Y or N
grade	Y or N	Y or N	Y or N
outcomes	Y or N	Y or N	Y or N
assessment result	Y or N	Y or N	Y or N
outcomes	Y or N	Y or N	Y or N
section_result	Y or N	Y or N	Y or N
section result	Y or N	Y or N	Y or N
outcomes	Y or N	Y or N	Y or N
item_result	Y or N	Y or N	Y or N
item result	Y or N	Y or N	Y or N
response	Y or N	Y or N	Y or N
outcomes	Y or N	Y or N	Y or N

Completion of the three columns is intended to reflect:

• Publish - this implies that the XML instance contains the identified elements. If such an element is not ticked then it will not occur within the exported QTI-XML instance(s);
• Accept - it is assumed that the ability to accept the contents of an element is accompanied by the ability to use, and if appropriate, display that content. If this is not the case but the content of the material can be exported then the 'Repackage' column can still be ticked;
• Repackage - this is the ability to import QTI-XML instances from one or more sources and to create a new instance that combines the imported information. It is not necessary for the repackaging system to be able to operate on the information supplied.

7.3 Interoperability Statement

An example of the detailed 'Interoperability Statement' is shown in Tables 7.2a, 7.2b, 7.2c, 7.2d and 7.2e (one for each of the core structures). Compliance to QTI means that at least one of the columns must be completed.

 

Table 7.2a Interoperability statement (context).

context		Version 1.2
Mandatory: This structure is mandatory i.e. it MUST be used in any 1EdTech QTI Results Reporting XML instance.
Optional Fields: Optional fields are informative. Checking an optional field implies that all of the associated mandatory elements are supported.
	Publish (export, data)	Accept (import, display)	Repackage Feature
qti_comment	q	q	q
name	q	q	q
generic_identifier	q	q	q
date	q	q	q
Extension Fields: These features allow the data model to be extended.
	Publish (export, data)	Accept (import, display)	Repackage Feature
extension_context	q	q	q

 

Table 7.2b Interoperability statement (summary_result).

summary result		Version 1.2
Optional Fields: Optional fields are informative. Checking an optional field implies that all of the associated mandatory elements are supported.
	Publish (export, data)	Accept (import, display)	Repackage Feature
type_label	q	q	q
generic_identifier	q	q	q
asi_title	q	q	q
date	q	q	q
status	q	q	q
duration	q	q	q
score	q	q	q
grade	q	q	q
outcomes	q	q	q
status	q	q	q
score	q	q	q
grade	q	q	q
Extension Fields: These features allow the data model to be extended.
	Publish (export, data)	Accept (import, display)	Repackage Feature
extension_summary_result	q	q	q

 

Table 7.2c Interoperability statement (assessment_result).

assessment result		Version 1.2
Optional Fields: Optional fields are informative. Checking an optional field implies that all of the associated mandatory elements are supported.
	Publish (export, data)	Accept (import, display)	Repackage Feature
qti_comment	q	q	q
asi_title	q	q	q
ident_ref	q	q	q
asi_metadata	q	q	q
asi_description	q	q	q
date	q	q	q
duration	q	q	q
objective	q	q	q
control	q	q	q
outcomes	q	q	q
status	q	q	q
score	q	q	q
grade	q	q	q
feedback_displayed	q	q	q
num_items	q	q	q
num_sections	q	q	q
num_items_presented	q	q	q
num_items_attempted	q	q	q
num_sections_presented	q	q	q
section_result	q	q	q
Extension Fields: These features allow the data model to be extended.
	Publish (export, data)	Accept (import, display)	Repackage Feature
extension_assessment_result	q	q	q
extension_score	q	q	q
extension_grade	q	q	q

 

Table 7.2d Interoperability statement (section_result).

section result		Version 1.2
Optional Fields: Optional fields are informative. Checking an optional field implies that all of the associated mandatory elements are supported.
	Publish (export, data)	Accept (import, display)	Repackage Feature
qti_comment	q	q	q
asi_title	q	q	q
ident_ref	q	q	q
presented	q	q	q
asi_metadata	q	q	q
asi_description	q	q	q
date	q	q	q
duration	q	q	q
objective	q	q	q
control	q	q	q
outcomes	q	q	q
status	q	q	q
score	q	q	q
grade	q	q	q
feedback_displayed	q	q	q
num_items	q	q	q
num_sections	q	q	q
num_items_presented	q	q	q
num_items_attempted	q	q	q
num_sections_presented	q	q	q
section_result	q	q	q
item_result	q	q	q
Extension Fields: These features allow the data model to be extended.
	Publish (export, data)	Accept (import, display)	Repackage Feature
extension_section_result	q	q	q
extension _score	q	q	q
extension _grade	q	q	q

 

Table 7.2e Interoperability statement (item_result).

item result		Version 1.2
Optional Fields: Optional fields are informative. Checking an optional field implies that all of the associated mandatory elements are supported.
	Publish (export, data)	Accept (import, display)	Repackage Feature
qti_comment	q	q	q
asi_title	q	q	q
ident_ref	q	q	q
presented	q	q	q
asi_metadata	q	q	q
asi_description	q	q	q
date	q	q	q
duration	q	q	q
objective	q	q	q
control	q	q	q
response	q	q	q
responseform	q	q	q
num_attempts	q	q	q
response_value	q	q	q
outcomes	q	q	q
status	q	q	q
score	q	q	q
grade	q	q	q
feedback_displayed	q	q	q
Extension Fields: These features allow the data model to be extended.
	Publish (export, data)	Accept (import, display)	Repackage Feature
extension_item_result	q	q	q
extension_responseform	q	q	q
extension_response	q	q	q
extension_score	q	q	q
extension_grade	q	q	q

Note that the 'Interoperability Statement' addresses support for the various elements within the binding. The set of attributes are not considered. Inclusion of conformance with respect to attributes will be considered in later versions of the specification.

It is important that the 'Interoperability Statement' is clear in showing what is and, perhaps more importantly, what is not supported. The usage of descriptive conformance approach has been adopted to encourage vendors to be as clear as possible when describing the capabilities of their QTI-compliant systems.

About This Document

Title	1EdTech Question & Test Interoperability Results Reporting Information Model
Editors	Colin Smythe, Lane Brewer, and Steve Lay
Version	1.2
Version Date	11 February 2002
Status	Final Specification
Summary	This document presents the 1EdTech QTI Results Reporting Information Model. This specification describes the data model for the exchange of results. These results may or may not have been derived from assessments that conform to the 1EdTech QTI ASI specification.
Revision Information	22 January 2002
Purpose	Defines the final specification for the results reporting part of the 1EdTech QTI specifications.
Document Location	http://www.imsglobal.org/question/v1p2/imsqti_res_infov1p2.html

List of Contributors

The following individuals contributed to the development of this document:

Russell Almond	ETS, USA
Lane Brewer	Galton Technologies Inc.
Todd Brewer	Galton Technologies Inc.
Russell Grocott	Can Studios Ltd.
Andy Heath	CETIS/JISC, UK
Paul Hilton	Can Studios Ltd.
John Kleeman	Question Mark Computing Ltd.
Steven Lay	University of Cambridge Local Examinations Syndicate, UK
Jez Lord	Can Studios Ltd.
Paul Roberts	Question Mark Computing Ltd.
Nial Sclater	CETIS/University of Strathclyde, UK
Colin Smythe	Dunelm Services Ltd.

Revision History

Version No.	Release Date	Comments
Base Document Version 1.2	7 June 2001	The first formal release of the 1EdTech QTI: Results Reporting Information Model Base Document.
Public Draft Version 1.2	30 September 2001	The formal public draft release of the 1EdTech QTI: Results Reporting Information Model Public Draft Specification Document. This is the first release of this document for public review.
Final Version 1.2	11 February 2002	The first final release of the V1.2 1EdTech QTI Results Reporting Information Model.

Index

A
ASI 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11
     Assessment 1, 2, 3, 4, 5, 6, 7
     Item 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
     Section 1, 2, 3, 4, 5, 6, 7, 8
Assessment 1, 2, 3, 4, 5, 6, 7
Assessment_result Elements
     assessment_result 1, 2, 3, 4, 5, 6
     extension_ assessment_result 1
Attributes
     asi_title 1, 2, 3, 4, 5, 6, 7, 8
     cardinality 1
     entityref 1, 2, 3, 4, 5, 6
     feedback_switch 1
     hint_switch 1
     ident_ref 1, 2, 3, 4, 5, 6, 7, 8, 9
     members 1
     presented 1, 2, 3, 4, 5, 6, 7, 8, 9
     render_type 1
     response_status 1
     response_time 1
     response_type 1
     solution_switch 1
     source 1
     status 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
     timing 1
     uri 1, 2, 3, 4, 5, 6, 7
     varname 1, 2
     vartype 1
     view 1
     xmllang 1
 

C
Common Elements
     asi_metadata 1, 2, 3, 4, 5, 6, 7, 8, 9
     control 1, 2, 3, 4, 5, 6, 7, 8, 9
     date 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
     duration 1, 2, 3, 4, 5, 6, 7, 8, 9
     feedback_displayed 1, 2, 3, 4, 5, 6, 7
     grade 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11
     grade_cut 1
     grade_value 1
     num_items 1, 2, 3, 4, 5
     num_items_attempted 1, 2, 3, 4, 5
     num_items_presented 1, 2, 3, 4, 5
     num_sections 1, 2, 3, 4, 5
     num_sections_presented 1, 2, 3, 4, 5
     other 1
     outcomes 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11
     qti_comment 1, 2, 3, 4, 5, 6, 7, 8, 9
     score 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14
     score_average 1
     score_cut 1
     score_interpretation 1
     score_max 1
     score_min 1
     score_normalized 1
     score_reliability 1
     score_value 1
     status 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
Context Elements
     context 1, 2, 3, 4, 5, 6, 7
     date 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
     extension_context 1, 2
     generic_identifier 1, 2, 3, 4, 5, 6
     name 1, 2, 3, 4, 5, 6, 7, 8, 9
 

D
DTD 1, 2

E
Elements
     asi_description 1, 2, 3, 4, 5, 6, 7
     asi_metadata 1, 2, 3, 4, 5, 6, 7, 8, 9
     asi_metadatafield 1, 2
     assessment_result 1, 2, 3, 4, 5, 6
     context 1, 2, 3, 4, 5, 6, 7
     control 1, 2, 3, 4, 5, 6, 7, 8, 9
     correct_response 1
     date 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
     datetime 1, 2
     duration 1, 2, 3, 4, 5, 6, 7, 8, 9
     extension_assessment_result 1
     extension_context 1, 2
     extension_grade 1, 2
     extension_item_result 1, 2
     extension_response 1, 2
     extension_responseform 1
     extension_score 1, 2
     extension_section_result 1
     extension_summary_result 1
     feedback_display 1, 2, 3, 4, 5, 6, 7
     field_name 1, 2
     field_value 1, 2
     generic_identifier 1, 2, 3, 4, 5, 6
     grade 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11
     grade_cut 1
     grade_value 1
     identifier_string 1
     item_result 1, 2, 3, 4, 5, 6, 7, 8
     name 1, 2, 3, 4, 5, 6, 7, 8, 9
     num_attempts 1, 2
     num_items 1, 2, 3, 4, 5
     num_items_attempted 1, 2, 3, 4, 5
     num_items_presented 1, 2, 3, 4, 5
     num_sections_presented 1, 2, 3, 4, 5
     objective 1, 2, 3, 4, 5, 6, 7
     other 1
     outcomes 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11
     qti_comment 1, 2, 3, 4, 5, 6, 7, 8, 9
     qti_result_report 1
     response 1, 2, 3, 4, 5, 6, 7
     response_value 1, 2
     result 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
     score 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14
     score_average 1
     score_cut 1
     score_interpretation 1
     score_max 1
     score_min 1
     score_normalized 1
     score_reliability 1
     score_std_error 1
     score_value 1
     section_result 1, 2, 3, 4, 5, 6, 7, 8
     status 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
     status_value 1
     summary_result 1, 2, 3, 4, 5, 6, 7
     type_label 1, 2, 3, 4, 5
     vocabulary 1, 2, 3, 4, 5, 6, 7, 8
Extension Elements
     extension 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17
     extension_assessment_result 1
     extension_context 1, 2
     extension_grade 1, 2
     extension_item_result 1, 2
     extension_response 1, 2
     extension_responseform 1
     extension_score 1, 2
     extension_section_result 1
     extension_summary_result 1
 

I
Item 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
Item_result Elements
     asi_metadata 1, 2, 3, 4, 5, 6, 7, 8, 9
     control 1, 2, 3, 4, 5, 6, 7, 8, 9
     correct_response 1
     date 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
     duration 1, 2, 3, 4, 5, 6, 7, 8, 9
     extension_response 1, 2
     item_result 1, 2, 3, 4, 5, 6, 7, 8
     num_attempts 1, 2
     objective 1, 2, 3, 4, 5, 6, 7
     outcomes 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11
     qti_comment 1, 2, 3, 4, 5, 6, 7, 8, 9
     response 1, 2, 3, 4, 5, 6, 7
     response_value 1, 2
 

M
Meta-data 1, 2, 3, 4, 5, 6, 7
     Description 1
 

S
Section 1, 2, 3, 4, 5, 6, 7, 8
Section_result Elements
     extension_section_result 1
Summary_result Elements
     summary_result 1, 2, 3, 4, 5, 6, 7
 

V
Vocabularies 1, 2

X
XML 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
     Schema
 

DTD 1, 2

XSD 1 XSD 1

 

 

 

1EdTech Consortium, Inc. ("1EdTech") is publishing the information contained in this 1EdTech Question & Test Interoperability: Results Reporting Information Model ("Specification") for purposes of scientific, experimental, and scholarly collaboration only.

1EdTech makes no warranty or representation regarding the accuracy or completeness of the Specification.
This material is provided on an "As Is" and "As Available" basis.

The Specification is at all times subject to change and revision without notice.

It is your sole responsibility to evaluate the usefulness, accuracy, and completeness of the Specification as it relates to you.

1EdTech would appreciate receiving your comments and suggestions.

Please contact 1EdTech through our website at http://www.imsglobal.org

Please refer to Document Name: 1EdTech Question & Test Interoperability: Results Reporting Information Model Date: 11 February 2002

---