1EdTech Question & Test Interoperability Metadata and Usage Data
Version: 2.1 Final
Date Issued: 31 August 2012
Latest version: http://www.imsglobal.org/question/
IPR and Distribution Notices
Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the specification set forth in this document, and to provide supporting documentation.
1EdTech takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on 1EdTech's procedures with respect to rights in 1EdTech specifications can be found at the 1EdTech Intellectual Property Rights web page: http://www.imsglobal.org/ipr/imsipr_policyFinal.pdf.
Copyright © 2005-2012 1EdTech Consortium. All Rights Reserved.
Use of this specification to develop products or services is governed by the license with 1EdTech found on the 1EdTech website: http://www.imsglobal.org/license.html.
Permission is granted to all parties to use excerpts from this document as needed in producing requests for proposals.
The limited permissions granted above are perpetual and will not be revoked by 1EdTech or its successors or assigns.
THIS SPECIFICATION IS BEING OFFERED WITHOUT ANY WARRANTY WHATSOEVER, AND IN PARTICULAR, ANY WARRANTY OF NONINFRINGEMENT IS EXPRESSLY DISCLAIMED. ANY USE OF THIS SPECIFICATION SHALL BE MADE ENTIRELY AT THE IMPLEMENTER'S OWN RISK, AND NEITHER THE CONSORTIUM, NOR ANY OF ITS MEMBERS OR SUBMITTERS, SHALL HAVE ANY LIABILITY WHATSOEVER TO ANY IMPLEMENTER OR THIRD PARTY FOR ANY DAMAGES OF ANY NATURE WHATSOEVER, DIRECTLY OR INDIRECTLY, ARISING FROM THE USE OF THIS SPECIFICATION.
Join the discussion and post comments on the QTI Public Forum: http://www.imsglobal.org/community/forum/categories.cfm?catid=52
The 1EdTech Logo is a trademark of the 1EdTech Consortium, Inc. in the United States and/or other countries.
Document Name: 1EdTech Question & Test Interoperability (QTI) Metadata and Usage Data Final v2.1 Revision: 31 August 2012
Table of Contents
- 1. Introduction
- 2. References
- 3. New metadata elements in 1EdTech QTI v2.0
- 4. IEEE LOM Profile
-
- 4.1. General
-
- 4.1.1. identifier
- 4.1.2. title
- 4.1.3. language
- 4.1.4. description
- 4.1.5. keyword
- 4.1.6. coverage
- 4.2. Lifecycle
-
- 4.2.1. version
- 4.2.2. status
- 4.2.3. contribute
- 4.3. meta_metadata
-
- 4.3.1. identifier
- 4.3.2. contribute
- 4.3.3. metadata_schema
- 4.3.4. language
- 4.4. Technical
-
- 4.4.1. format
- 4.4.2. size
- 4.4.3. location
- 4.4.4. Other Platform Requirements
- 4.5. Educational
-
- 4.5.1. learning_resource_type
- 4.5.2. Context
- 4.5.3. typical_learning_time
- 4.5.4. description
- 4.5.5. language
- 4.6. Rights
-
- 4.6.1. cost
- 4.6.2. copyright_and_other_restrictions
- 4.6.3. description
- 4.7. Relation
- 4.8. Annotation
- 4.9. Classification
- 5. Usage Data
- 6. XML Binding
1. Introduction
Previous versions of the 1EdTech QTI specification had a specific metadata set contained within the data structures themselves, i.e. the ASI. That metadata vocabulary had its own set of names, all of which started with the characters 'qmd_'.
In QTI version 2.0, QTI-specific metadata was brought into line with the IEEE LOM in accordance with the 1EdTech Metadata Best Practice and Implementation Guide for [LOM]. The IEEE LOM standard defines a set of metadata elements that can be used to describe learning resources, but does not describe assessment resources in sufficient detail. The application profile provided in this document therefore extends the IEEE LOM to meet the specific needs of QTI developers wishing to associate metadata with items (as defined by the accompanying Item Information Model). QTI version 2.1 further extends this to enable the description of tests, pools and object banks.
2. References
- 1EdTech Learning Resource Metadata XML Binding, Version 1.2.1
- LOM
- IEEE 1484.12.1-2002 Standard for Learning Object Metadata (LOM)
- RDN
- RDN/LTSN resource type vocabulary
- http://www.rdn.ac.uk/publications/rdn-ltsn/types/
- VDEX
- 1EdTech Vocabulary Definition Exchange, Version 1.0, February 2004
- http://www.imsglobal.org/vdex/index.html
3. New Metadata Elements in 1EdTech QTI v2
The IEEE LOM permits extensions to be made to the conceptual data schema, in the form of new terms for existing vocabularies, new vocabularies for existing elements or new elements, which may be inserted into the schema provided they do not subvert the existing chain of references or introduce new data types for existing fields.
It should be noted that extensions are community specific and will impact significantly on the interoperability of the metadata which contains them.
Secondary metadata, sometimes known as 'usage data' (item statistics), is defined separately in its own data model. See Usage Data later in this document.
The following class describes a new category of metadata for the recording of QTI specific information. It is designed to be treated as an additional top-level category to augment the LOM profile described in the next section.
Contains : itemTemplate boolean [0..1]
True if the item is actually an item template, in other words, the item changes its appearance based on some random or external factor. An assessmentItem that contains a templateProcessing section.
Contains : timeDependent boolean [0..1]
Whether or not the item is time dependent. A time dependent item takes the length of time taken for an attempt into consideration when scoring.
Contains : composite boolean [0..1]
True if the item comprises more than one interaction, for example, an assessmentItem that contains more than one interaction.
Contains : interactionType [*]
The interaction type(s) of the item. The vocabulary is comprised of the names, as defined in the information model, of the leaf classes derived from interaction.
Contains : feedbackType [0..1]
Describes the type of feedback, if any, available in the item. If feedback is available then it is described as being non-adaptive or adaptive depending on whether the item is itself adaptive. A non-adaptive item generates feedback based on the responses submitted as part of (the last) attempt only. An adaptive item generates feedback that takes into consideration the path taken through the item, in other words, feedback based on the accumulation of all attempts and not just the last.
Contains : solutionAvailable boolean [0..1]
Set to true if a model solution is available for the item. For example, an assessmentItem that provides correct responses for all declared response variables.
Contains : toolName string256 [0..1]
The name of the tool used to author the evaluation object.
Contains : toolVersion string256 [0..1]
The version of the tool used to author the evaluation object.
Contains : toolVendor string256 [0..1]
The company which produced the tool used to author the evaluation object.
none
No feedback is available.
nonadaptive
Feedback is available but it is non-adaptive. In other words, the item is a non-adaptive item.
adaptive
Feedback is available and is adaptive. In other words, the item is an adaptive item.
4. IEEE LOM Profile
QTI v2.0 deprecated use of the relation category when creating metadata instances for QTI items, reserving it for future use. A number of use cases which informed the work on v2.1 raised scenarios which could be best addressed through the use of this category.
4.1. General
Note that the LOM-defined Structure and AggregationLevel fields are not recommended by this profile.
4.1.1. identifier
One of the values given for the identifier must have an entry that matches the identifier of the associated item, test or object bank.
4.1.2. title
The title must have a value that matches the value of the title attribute of the associated item, test or object bank. The language used to interpret the title is taken from the lang attribute of the associated item, test or object bank.
4.1.3. language
There must be one value for each of the languages referred to by the language attributes on the associated assessmentItem, test or object bank and its bodyElements.
4.1.4. description
When transforming item metadata records with no description into systems that require a value for this field, the title of the item, test or object bank should be used to set the value of the description.
4.1.5. keyword
When transforming item metadata records with no keywords into systems that require a value for this field, the title of the item, test or object bank should be used to derive a set of keywords. It should be noted that LOM specifically indicates the use of classification for the description of subject area, and specifies that general.keyword 'should not be used for characteristics that can be described by other data elements'.
4.1.6. coverage
Usage as defined by [LOM].
4.2. Lifecycle
4.2.1. version
See comment in status below for important information about the use of this field.
4.2.2. status
It is anticipated that systems for handling assessment resources would prefer to draw from wider vocabularies than the one defined by [LOM]. However, in order to facilitate the transformation of metadata instances to systems that require the use of the LOM vocabulary for this field it is recommended that the version is used to achieve the tracking of items, tests and object banks through more complex production processes. The classification category may also be used to express the status and availability of assessment resources.
4.2.3. contribute
Usage as defined by [LOM].
4.3. meta_metadata
4.3.1. identifier
A globally unique label that identifies this metadata record.
4.3.2. contribute
Usage as defined by [LOM].
4.3.3. metadata_schema
Metadata records that adhere to this profile are conforming LOM instances, therefore references to both this specification and LOM are applicable. The appropriate references are IMSQTIv2.1 and LOMv1.0. References to other schemas to which the metadata instance conforms are also permitted.
4.3.4. language
There are two approaches to providing multilingual information in LOM-based metadata records which can be used separately or in combination. The first is to translate the metadata on a field-by-field basis providing each field value as a set of strings, each individually language tagged. The alternative is to generate multiple equivalent metadata records and use this language field (on the meta-metadata category) to set the default language for the whole record. This profile prefers the latter approach, metadata records conforming to this profile should not provide multilingual values to individual fields within the record.
4.4. Technical
Note that the LOM-defined Requirement, Installation Remarks and Duration fields are not recommended by this profile.
4.4.1. format
There should be at least one instance of format with the value text/x-imsqti-item-xml, text/x-imsqti-test-xml or application/xml .
4.4.2. size
Usage as defined by [LOM].
4.4.3. location
Usage as defined by [LOM].
4.4.4. Other Platform Requirements
Usage as defined by [LOM].
4.5. Educational
Note that the LOM-defined Interactivity Type, Interactivity Level, Semantic Density, Intended End User Role, Typical Age Range and Difficulty fields are not recommended by this profile.
4.5.1. learning_resource_type
QTI objects are designed to be reusable in a variety of assessment scenarios. Therefore, the LOM-defined values self assessment and exam are forbidden. If the standard LOM vocabulary is used then only the values exercise or questionnaire should be used to describe an item. An alternative vocabulary for this field has been defined in [RDN], when using that vocabulary the value AssessmentItem, ExaminationTest and QuestionBank are recommended.
4.5.2. Context
This is used to provide an educational context for the value given in typical learning time.
4.5.3. typical_learning_time
In the context of a QTI object, the typical learning time is interpreted as the length of time the candidate would normally be allocated to complete the object. It is not a time limit, however, when building a time-limited test from an item bank the typical learning times of the selected items may be added together to estimate the expected duration of the test and used to calculate a time limit for the test if required.
4.5.4. description
Item, test or pool objectives should be included in this field if required.
4.5.5. language
Usage as defined by [LOM].
4.6. Rights
Note that the LOM-defined cost and copyright_and_other_restrictions fields are highly problematic, and that a more detailed rights description language is necessary to adequately express the often complex rights issues surrounding resource creation and reuse. However, the vast majority of application profiles mandate the rights category, and it is therefore recommended for use within this application profile.
Care should be taken when an item depends on a (shared) media file with its own metadata to ensure that restrictions on the use of the media file are reflected in the overal rights description of the item itself.
4.6.1. cost
Usage as defined by [LOM].
4.6.2. copyright_and_other_restrictions
Usage as defined by [LOM].
4.6.3. description
Usage as defined by [LOM].
4.7. Relation
The LOM relation category is used to describe the relationship between learning objects, and was reserved in QTI v2.0 for future use. In v2.1, this category is used to express the relationship between items and tests, fragments and the objects that include them and individual relationships between items.
4.7.1. kind
A number of relationships between items may be described without extending the LOM vocabulary. However, a new vocabulary element has been introduced to allow one of the more complex inter-item relationships commonly expressed in item banks. In addition, interpretations of some LOM vocabulary elements are provided. It should be noted that each target should have a new relationship instance.
| Element name | Present in LOM? | Explanation |
|---|---|---|
| ispartof | Y | A fragment may refer to the objects which include it using this term. The relation.resource.identifier containing the identifier(s) of the including objects. |
| haspart | Y | An object my refer to the fragments which it includes using this term. The relation.resource.identifer containing the fragment's identifier(s). |
| isversionof | Y | Usage as defined by [LOM]. |
| hasversion | Y | Usage as defined by [LOM]. |
| isformatof | Y | Usage as defined by [LOM]. |
| hasformat | Y | Usage as defined by [LOM]. |
| references | Y | Usage as defined by [LOM]. |
| isreferencedby | Y | Usage as defined by [LOM]. |
| isbasedon | Y | Usage as defined by [LOM]. |
| isbasisfor | Y | Usage as defined by [LOM]. |
| requires | Y | Usage as defined by [LOM]. Note that this is the appropriate way to represent the relationship between a test or test fragment and the items that it refers to. |
| isrequiredby | Y | Usage as defined by [LOM]. Note that this is the appropriate way to represent the relationship between an item and the tests or test fragments that refer to it. |
| precludes | N | This term can be used to indicate items which must not be incorporated into the same test as each other (sometimes referred to as enemy items). This relationship is symmetric. |
4.7.2. resource
Usage as defined by [LOM].
4.8. Annotation
Usage as defined by [LOM].
4.9. Classification
Usage as defined by [LOM]. The preferred solution for the description of an item_s subject area is to used the LOM classification category with the value classification.purpose = "discipline". This allows the use of any recognized or bespoke subject classification scheme such as the top level(s) of the Library of Congress Classification (LCC) and Dewey Decimal Classification (DDC), or institutional, regional or national curriculum classifications, as specified under classiciation.taxonPath.source. For more detailed description of topics within subject areas, the value classification.purpose = "idea" may be used with further levels of LCC, DDC or subject-specific classification schemes. The classification category may also be used to describe the visibility and availability of items beyond the limited vocabulary provided by LOM lifecycle.status. The use of custom schemes for classification enables repository administrators to capture all the information they need to capture, using the terms most appropriate for that institution. There may be multiple instances of the classification category, enabling detailed classification of assessments by subject area and association of an assessment with a number of different subject areas or topics.
5. Usage Data
Usage data, most commonly item statistics, do not form part of an assessmentItem directly because they always relate to some context or domain in which the statistics are valid. Therefore, this specification defines a separate class for describing these statistics.
Each statistic refers to both its context and to the assessmentItem(s) it relates to. Therefore, instances of this class are bound and packaged separately for interoperability.
Attribute : glossary [0..1]: uri
An optional URI that identifies the default glossary in which the names of the itemStatistics are defined.
Abstract class : itemStatistic
- Derived classes:
categorizedStatistic, ordinaryStatistic- Associated classes:
usageData
A value or set of values that describe the performance of the item within a specific context. Common measures include the item's difficulty and how well it discriminates between various candidate ability levels
Attribute : name [1]: identifier
The unique identifier of the item statistic. Glossaries of identifiers defined by this specification for commonly used item and distractor statistics are defined and should be used where possible. See Vocabulary for the Exchange of Item Statistics for more details.
Attribute : glossary [0..1]: uri
An optional URI that identifies the glossary in which the name is defined. This value overrides any default glossary provided by the glossary attribute of the parent usageData.
Attribute : context [1]: uri
A Uniform Resource Identifier that points to information about the context within which the item statistic was created. For example, the URI may point to the sample of item scores and the specifics of computations that created item statistics. The URI may be a URL, a database index, or other valid identifier
Attribute : caseCount [0..1]: integer
The number of cases in the sample used to create the item statistic.
Attribute : stdError [0..1]: float
The standard error of the item statistic, also known as the variance.
Attribute : stdDeviation [0..1]: float
The standard deviation of the item statistic (i.e. the square root of the standard error).
Attribute : lastUpdated [0..1]: date
Date of the last update to the item statistic value.
Contains : targetObject [1..*]
The targetObject is used to refer to an assessment object. This object may be an assessmentItem or some other type of object defined outside the scope of this specification, for example, an entire test. In some cases it is desirable to refer not just to the assessment object but to a specific part of that object, in which case the optional partIdentifier can be used.
Attribute : identifier [1]: string
The identifier of the assessmentItem or other target object.
Attribute : partIdentifier [0..1]: identifier
An optional identifier to a specific part (e.g. an itemVariable) defined within the assessment object. In the case of an assessmentItem the partIdentifier typically refers to an outcome variable but can refer to other objects identified in the same namespace, such as a specific choice within an interaction. If no partIdentifier is given the statistic is considered to refer to the target object as a whole.
An item statistic that consists of a single numeric value.
Contains : value [1]
An item statistic that consists of multiple values, e.g. IRT Dispersion Parameters.
Contains : mapping [1]
5.1. Vocabulary for the Exchange of Item Statistics
This specification defines a vocabulary to aid the exchange of commonly used statistics. The vocabulary is split into two glossaries.
Item Statistics
glossaries/item_statistics.xml
The main item statistics glossary defines statistics that refer to a specific outcome of an item (typically the outcome variable SCORE).
Distractor Statistics
glossaries/distractor_statistics.xml
The distractor statistics glossary defines statistics that refer to a specific response (typically a simpleChoice) within an item.
These vocabularies have been defined using [VDEX]. It is recongized that vocabularies may differ widely across application areas. Users of this specification are encouraged to document and share their own vocabularies using this common format.
6. XML Binding
The accompanying XML binding provides a binding for the qtiMetadata object that is consistent with the binding given in [IMS_MD_Binding]. The qtiMetadata class defines a new category that could appear alongside LOM categories such as General, Lifecycle, etc. In the context of the 1EdTech binding, that means it would naturally appear as a direct descendant of the <lom> object itself. The 1EdTech binding does not support extension at this point in the XML binding however, so qtiMetadata is bound separately and must be used in parallel to the LOM object as an additional metadata object.
At the time of writing, the IEEE itself is working on a binding document for [LOM]. In the future, an XML binding more consistent with the approach taken by the IEEE is expected to be recommended.
About This Document
| Title | 1EdTech Question & Test Interoperability Metadata and Usage Data |
| Editors | Wilbert Kraan (JISC/CETIS), Steve Lay (Cambridge Assessment), Pierre Gorissen (SURF) |
| Version | Final v2.1 |
| Version Date | 31 August 2012 |
| Status | Final Release Specification |
| Summary | This document describes the QTI metadata and usage data used for test, pools, and object banks. |
| Revision Information | 31 August 2012 |
| Purpose | This document has been approved by the 1EdTech Technical Advisory Board and is made available for adoption and conformance. |
| Document Location | http://www.imsglobal.org/question/qtiv2p1/imsqti_mdudv2p1.html |
| To register any comments or questions about this specification please visit: http://www.imsglobal.org/community/forum/categories.cfm?catid=52 | |
List of Contributors
The following individuals contributed to the development of this document:
| Name | Organization |
|---|---|
| Odette Auzende | Université Pierre et Marie Curie (France) |
| Dick Bacon | JISC/CETIS (UK) |
| Niall Barr | University of Glasgow/1EdTech (UK) |
| Lance Blackstone | Pearson (USA) |
| Jeanne Ferrante | ETS (USA) |
| Helene Giroire | Université Pierre et Marie Curie (France) |
| Pierre Gorissen | SURF (The Netherlands) |
| Regina Hoag | ETS (USA) |
| Wilbert Kraan | JISC/CETIS (UK) |
| Gopal Krishnan | Pearson (USA) |
| Young Jin Kweon | KERIS (South Korea) |
| Steve Lay | Cambridge Assessment (UK) |
| Francoise LeCalvez | Université Pierre et Marie Curie (France) |
| David McKain | JISC/CETIS (UK) |
| Mark McKell | 1EdTech (USA) |
| Sue Milne | JISC/CETIS (UK) |
| Jens Schwendel | BPS Bildungsportal Sachsen GmbH (Germany) |
| Graham Smith | JISC/CETIS (UK) |
| Colin Smythe | 1EdTech (UK) |
| Yvonne Winkelmann | BPS Bildungsportal Sachsen GmbH (Germany) |
| Rowin Young | JISC/CETIS (UK) |
Revision History
| Version No. | Release Date | Comments |
|---|---|---|
| Base Document 2.1 | 14 October 2005 | The first version of the QTI v2.1 specification. |
| Public Draft 2.1 | 9 January 2006 | The Public Draft v2.1 of the QTI specification. |
| Public Draft 2.1 (revision 2) | 8 June 2006 | The Public Draft v2.1 (revision 2) of the QTI specification. |
| Final Release v2.1 | 31 August 2012 | The Final Release v2.1 of the QTI specification. Includes updates, error corrections, and additional details. |
1EdTech Consortium, Inc. ("1EdTech") is publishing the information contained in this 1EdTech Question and Test Interoperability Metadata and Usage Data ("Specification") for purposes of scientific, experimental, and scholarly collaboration only.
1EdTech makes no warranty or representation regarding the accuracy or completeness of the Specification.
This material is provided on an "As Is" and "As Available" basis.
The Specification is at all times subject to change and revision without notice.
It is your sole responsibility to evaluate the usefulness, accuracy, and completeness of the Specification as it relates to you.
1EdTech would appreciate receiving your comments and suggestions.
Please contact 1EdTech through our website at http://www.imsglobal.org
Please refer to Document Name: 1EdTech Question and Test Interoperability Metadata and Usage Data Revision: 31 August 2012
