1EdTech Accessible Portable Item Protocol™ (APIP™): Conformance and Certification

 

Final Specification
Version 1.0

 

Date Issued:             31 March 2014
Latest version:         http://www.imsglobal.org/apip/index.html

IPR and Distribution Notices

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the specification set forth in this document, and to provide supporting documentation.

1EdTech draws attention to the fact that it is claimed that compliance with this specification may involve the use of Measured Progress U.S. patent 8,303,309. 1EdTech takes no position concerning the evidence, validity or scope of such patent rights. Measured Progress has assured 1EdTech that it is willing to license patent rights it owns or controls which would necessarily be infringed by any implementation of this specification to those licensees (Members and non-Members alike) desiring to implement this specification. The statement of Measured Progress to such effect has been filed with 1EdTech. Information regarding a Reasonable and Non-Discriminatory - Zero cost (RAND-Z) license may be obtained from Measured Progress at http://www.measuredprogress.org/ipr-apip.

Attention is also drawn to the possibility that some of the elements of this specification may be the subject of patent rights other than those identified above. 1EdTech shall not be responsible for identifying any or all such patent rights. 1EdTech takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on 1EdTech's procedures with respect to rights in 1EdTech specifications can be found at the 1EdTech Intellectual Property Rights web page: http://www.imsglobal.org/ipr/imsipr_policyFinal.pdf.

Copyright © 2012-14 1EdTech Consortium. All Rights Reserved.

Use of this specification to develop products or services is governed by the license with 1EdTech found on the 1EdTech website: http://www.imsglobal.org/speclicense.html.

Permission is granted to all parties to use excerpts from this document as needed in producing requests for proposals.

The limited permissions granted above are perpetual and will not be revoked by 1EdTech or its successors or assigns.

THIS SPECIFICATION IS BEING OFFERED WITHOUT ANY WARRANTY WHATSOEVER, AND IN PARTICULAR, ANY WARRANTY OF NONINFRINGEMENT IS EXPRESSLY DISCLAIMED. ANY USE OF THIS SPECIFICATION SHALL BE MADE ENTIRELY AT THE IMPLEMENTER'S OWN RISK, AND NEITHER THE CONSORTIUM, NOR ANY OF ITS MEMBERS OR SUBMITTERS, SHALL HAVE ANY LIABILITY WHATSOEVER TO ANY IMPLEMENTER OR THIRD PARTY FOR ANY DAMAGES OF ANY NATURE WHATSOEVER, DIRECTLY OR INDIRECTLY, ARISING FROM THE USE OF THIS SPECIFICATION.
 

Public contributions, comments and questions can be posted here: Public APIP Forums

 

© 2014 1EdTech Consortium, Inc.
All Rights Reserved.

The 1EdTech Logo, Accessible Portable Item Protocol (APIP), and Question and TestInteroperability (QTI) are trademarks of the 1EdTech Consortium, Inc. in the United States and/or other countries.
Documents Name:  1EdTech APIP Conformance and Certification - Revision: 31 March 2014

Executive Summary

1EdTech APIP™ - Accessible Portable Item Protocol™ - is a technical standard for digital interchange of assessment content (items and tests). APIP also describes digital delivery options for a range of common accessibility needs. Use of APIP enables assessment materials to be exchanged digitally among a wide variety of products, such as item/test authoring products, item banks, and test delivery systems.

APIP addresses two long-standing needs in assessments. First, it allows for the transfer of content between vendors using a standardized XML format. Second, it supplies the necessary accessibility information in that content to support the needs of diverse  students. Using 1EdTech Access For All Personal Needs and Preferences (PNP) profile information, the content can be tailored to meet the access needs of each student.

The APIP standard defines an assessment file exchange that serves all students, not only students who have disabilities or are English language learners. In fact, the default content of the question (item) is based on the well established 1EdTech Question and Test Interoperability (QTI) standard that has been used internationally for over a decade. APIP uses a new profile of QTI v2.1.

Figure 1 illustrates the basic idea of APIP. Authors create content, providing extra information when needed to make that content accessible to all students. An assessment rostering system can provide the specific needs of individual students; these are documented in the PNP profile. An APIP certified delivery system takes the needs of a specific student, and ensures that the content is provided to the student with all their needed access supports.

 

Figure 1 The Basic Concept of APIP.

The three main areas of APIP are:

•         Content: The accessible content, which has supporting information for different kinds of accessibility needs

•         PNP: The user Personal Needs & Preferences profile, which tells the delivery system what the needs are for specific test takers

•         Delivery: The delivery system, which combines the user needs with the content to ensure the item is accessible for the test taker

 

1                  Introduction

1.1            Scope and Context

The purpose of this document is to define the different areas of conformance available for Accessible Portable Item Protocol (APIP) and to describe the procedures for achieving certification for version 1.0.  APIP is a standard created and governed by 1EdTech Consortium, a non-profit learning standards organization. This document is intended to be helpful, for example, to vendors who wish to certify their systems as officially conforming to APIP v1.0. Ultimately, certification of systems will help ensure that diverse end-users (students) receive access to computer-based assessments.

1.2            Status of this Document

This document is Candidate Final, meaning the technical solution described in the specification is as definitive as possible in the absence of numerous systems and content that have implemented the specification and achieved official conformance. 1EdTech strongly encourages its members and the community in general to continue/begin developing to the specification now. When a sufficient number of systems have proven the solution works through successful conformance achievement, the Candidate Final specification will be submitted to the 1EdTech membership for Final approval. 1EdTech expects only minor refinements based on implementation experience during this Candidate Final period.

1EdTech strongly encourages its members and the community to provide feedback to continue the evolution and improvement of the APIP standard. To join the 1EdTech developer and conformance certification community focused on APIP and QTI please visit the 1EdTech QTI/APIP Alliance online here: http://www.imsglobal.org/apip/index.html

Public contributions, comments and questions can be posted here: Public APIP Forums.

1.3            Structure of this Document

The structure of the rest of this document is:

2.  Conformance and Certification Overview	Describes how the conformance program works and the process for achieving certification;
3.  Assessment Conformance Statements	Contains details of the conformance options available for certification to the APIP v1.0 and the QTI v2.1 specifications;
4.  Achieving Certification	Describes the mechanics of the conformance program;
Appendix A - Accessibility Features	Lists the complete access features available in APIP v1.0 organized by categories;
Appendix B - RFP Language for APIP	Provides information about how to properly describe APIP requirements to be included in Requests for Proposals.

 

1.4            Related Documents

[APIP, 14a]     Accessible Portable Item Protocol (APIP) Best Practices and Implementation Guide v1.0 Final Specification, G.Driscoll, T.Hoffmann, W.Ostler, M.Russell, M.McKell and C.Smythe, 1EdTech Inc., March 2014.

[APIP, 14b]     Accessible Portable Item Protocol (APIP) Terms and Definitions v1.0 Final Specification, G.Driscoll, T.Hoffmann, W.Ostler, M.Russell, M.McKell and C.Smythe, 1EdTech Inc., March 2014.

[CC, 11]          1EdTech Common Cartridge Profile v1.2, J.Kahn, 1EdTech, Inc., October 2011.

[PNP, 10]        1EdTech Access For All Personal Needs & Preferences Information Model v2.0, R.Schwerdtfeger, M.Rothberg and C.Smythe, Final Release, 1EdTech Inc., April 2010.

[QTI, 12]         1EdTech Question & Test Interoperability Assessment Test, Section and Item Information Model v2.1 Final Specification, S.Lay and P.Gorissen, 1EdTech Inc., August 2012.

 

2                  Conformance and Certification Overview

2.1            Relationship between 1EdTech APIP and 1EdTech QTI

APIP uses a precisely defined subset of 1EdTech QTI v2.1 with additional features from some other 1EdTech specifications. The first step in implementing APIP is to implement the APIP defined subset of QTI v2.1. This will allow assessment suppliers to utilize QTI when accessibility is not a requirement and to receive a QTI conformance certification.

APIP and the profiles of QTI used in APIP will stay synchronized by 1EdTech in conjunction with 1EdTech member organizations through the QTI/APIP Accredited Profile Management Group (APMG). The QTI subset within APIP, such as interaction types (question types), is expected to grow over time as the marketplace for digital assessment evolves.

The additional features of APIP beyond QTI are accessibility metadata (based on 1EdTech Access For All), accessible delivery options (developed on the original APIP grant program), and curriculum standards metadata (based on 1EdTech Content Packaging/Common Cartridge). These features are defined in the APIP specification.

2.2            Conformance Certifications

1EdTech is establishing well-defined conformance certifications for QTI v2.1 and APIP v1.0 to ensure high levels of interoperability.  The conformance certification process involves testing for the ability to support different levels of features. The conformance details are described in Section 3.

2.3            How Conformance Certification is Achieved and Recorded

Conformance certification for a product is granted after passing a series of tests as described in the conformance instructions, submitting the results to 1EdTech, and agreeing to become part of the support community (the QTI/APIP Alliance http://www.imsglobal.org/apip/index.html). 1EdTech records all certifications on a public website: http://imscert.org. At the time conformance is certified, 1EdTech issues a unique registration number for the product name and version for use in proposals. Buyers and users are encouraged to visit the certification website to find certified products and to log any issues with certified products.

The community aspect of conformance is critical to address any issues that may arise in the marketplace as well as the evolution to new versions.  1EdTech requires suppliers to participate in resolving issues that may arise, as well as in helping to guide and implement each incremental revision to the conformance certifications.

2.4            Achieving Conformance Certification

The tests required for certification are different depending on the particular certification sought. The tests are also different for content/authoring tools versus delivery platforms. This is because content/authoring tools are tested to ensure that the content produced and exported is conformant within the scope of item types and tests that a particular tool can produce, while delivery systems are tested to ensure they can appropriately process a range of content alternatives. Conformant delivery platforms must import conformant content and provide conformant results. Conformant authoring tools must export conformant content and may or may not import conformant content, depending on the purpose of the tool.

3                  Assessment Conformance Statements

The following conformance matrix lists the current profiles available for QTI and APIP certification as they relate to these categories:
 

QTI Delivery System & Authoring System	The QTI conformance criteria for Delivery Systems and Authoring Systems.
APIP Authoring System	The APIP conformance criteria for Authoring Systems and APIP packages.
APIP Delivery System	The APIP conformance criteria for Delivery Systems.
APIP Personal Needs & Preferences (PNP) Systems	The APIP PNP conformance criteria for systems supplying or processing PNP instances.
APIP or QTI Item Bank	Conformance for a repository that can store any APIP Entry and/or APIP Core Item Packages (or QTI Item Packages, respectively) as defined as export from an Authoring System. The conformance statement should identify which profile(s) are supported (Core/Entry, etc.).
APIP or QTI Test Bank	Conformance for a repository that can store any APIP Entry and/or APIP Core Test Packages (of QTI Test Packages, respectively) as defined as export from an Authoring System. The conformance statement should identify which profile(s) are supported (Core/Entry, etc.).

3.1            Conformance Profiles

Named profiles currently defined for conformance are described here as they appear in the matrix below.
 

QTI v2.1 ASI Base	QTI Assessment, Section, Item Base conformance that offers general purpose certification comprising all features and interactions of QTI v2.1 as elective.
QTIv2.1 Summative Entry	Conformance for Delivery Systems that process Tests used for Summative assessment (this is equivalent to APIP Entry minus the accessibility features).
QTIv2.1 Formative Entry	Conformance for Delivery Systems that process Tests based upon QTIv2.1 (this is equivalent to the Common Cartridge profile for QTIv2.1).
QTIv2.1 Core	Conformance for Delivery Systems that process Tests based upon QTIv2.1 (this is equivalent to the Common Cartridge profile but with all of the prohibited features made Elective).
APIP Entry	The entry level APIP conformance (this is equivalent to the Common Cartridge profile). Entry level conformance also permits the set of implementation Electives.
APIP Core	This is the APIP Entry level plus the set of implementation Electives. It is to this level that 1EdTech recommends vendors aspire.
APIP PNP Entry	This is the APIP PNP Entry level requirements for a PNP System producing or processing PNP instances.
APIP PNP Core	This is the APIP PNP conformance level that 1EdTech recommends vendors aspire for a Preferences System producing or processing PNP instances.

 

Legend

[ X ]       Denotes that an implementation IS NOT expected to support the feature;

[ R ]       Denotes that an implementation MUST support the feature;

[ e ]        Denotes that an implementation MAY ELECT to support the feature;

[ N/A ]   Denotes that the feature is unavailable to this profile i.e. it is covered by another established profile;

[ e** ]    Delivery of Braille is somewhat non-standard at this time. Content and user profiles are required at the Core level to support Braille in anticipation of future delivery implementations.

All cells that are shaded (light red) denote functionalities for which 1EdTech conformance testing is not available in the initial release of the 1EdTech Assessment Conformance System; testing for one or more of these functionalities may become available later, even before a succeeding version of the APIP specification. Note that currently, all the shaded functionalities are elective. Conformance testing of systems will be through self-testing using an 1EdTech provided set of APIP packages. Conformance testing of content instances will be through the corresponding Online 1EdTech Validator and will ONLY address the validity of the XML with respect to the APIP specification; semantic integrity between the accessibility data and the original content will NOT be addressed.

 

 

APIP/QTI CONFORMANCE STATEMENTS
Id	Feature	QTI Delivery System & Authoring System				APIP Authoring System		APIP Delivery System		APIP PNP System
		QTIv2.1 ASI Base	QTIv2.1 Summative	QTIv2.1 Formative	QTIv2.1 Core	APIP Entry	APIP Core	APIP Entry	APIP Core	APIP PNP Entry	APIP PNP Core
Tests
T1	Outcome Declaration	e	R	R	R	R	R	R	R	N/A	N/A
T2	Time Limits	e	R	R	R	R	R	R	R	N/A	N/A
T2	Stylesheet	e	R	R	R	R	R	R	R	N/A	N/A
T4	Test Parts	e	R	R	R	R	R	R	R	N/A	N/A
T5	Item Session Control	e	R	R	R	R	R	R	R	N/A	N/A
T6	Time Limits	e	e	X	e	e	e	e	e	N/A	N/A
T7	Sections	e	R	R	R	R	R	R	R	N/A	N/A
T8	Test Feedback	e	X	R	R	X	e	X	e	N/A	N/A
T9a	Outcomes Processing-Full	e	e	X	e	e	R	e	R	N/A	N/A
T9b	Outcomes Processing-CC	e	R	R	R	R	X	R	X	N/A	N/A
T10	Test Feedback	e	e	R	R	e	e	e	e	N/A	N/A
Sections
S1	Item Session Control	e	e	X	e	e	e	e	e	N/A	N/A
S2	Time Limits	e	e	X	e	e	e	e	e	N/A	N/A
S3	Selection	e	X	X	e	X	e	X	e	N/A	N/A
S4	Ordering	e	X	X	e	X	e	X	e	N/A	N/A
S5	Rubric Block	e	R	R	R	R	R	R	R	N/A	N/A
S6	Include	e	e	X	e	e	e	e	e	N/A	N/A
S7	Items	e	R	R	R	R	R	R	R	N/A	N/A
S8	Sections	e	R	R	R	R	R	R	R	N/A	N/A
Interactions
Q1	Associate Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q2	Choice Interaction	e	R	R	R	R	R	R	R	N/A	N/A
Q3a	Custom Interaction (LTI)	e	e	e	e	e	e	e	e	N/A	N/A
Q3b	Custom Interaction (PCI)	e	e	X	e	e	e	e	e	N/A	N/A
Q4	Drawing Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q5	End Attempt Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q6	Extended Text Interaction	e	R	R	R	R	R	R	R	N/A	N/A
Q7	Gap Match Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q8	Graphic Associate Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q9	Graphic Gap Match Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q10	Graphic Order Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q11	Hot-spot Interaction	e	R	X	R	e	R	e	R	N/A	N/A
Q12	Hot-text Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q13	Inline Choice Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q14	Match Interaction	e	R	X	R	e	R	e	R	N/A	N/A
Q15	Media Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q16	Order Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q17	Position Object Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q18	Selection Point Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q19	Slider Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q20	Text Entry Interaction	e	R	R	R	R	R	R	R	N/A	N/A
Q21	Upload Interaction	e	e	X	e	e	e	e	e	N/A	N/A
Q22	Custom Interaction (proprietary)	e	X	X	X	e	e	X	e	N/A	N/A
Items
I1	responseDeclaration	e	R	R	R	R	R	R	R	N/A	N/A
I2	outcomeDeclaration	e	R	R	R	R	R	R	R	N/A	N/A
I3	templateDeclaration	e	e	X	e	e	e	e	e	N/A	N/A
I4	Shared Stimulus	e	e	e	e	e	e	e	e	N/A	N/A
I5	templateProcessing	e	e	X	e	e	e	e	e	N/A	N/A
I6	stylesheet	e	R	R	R	R	R	R	R	N/A	N/A
I7	Item Body	e	R	R	R	R	R	R	R	N/A	N/A
I8	HTML (Block)	e	R	R	R	R	R	R	R	N/A	N/A
I9a	Response Processing-Full	e	e	X	e	e	e	e	e	N/A	N/A
I9b	Response Processing-Fixed Template	e	R	R	R	R	R	R	R	N/A	N/A
I10	Modal Feedback	e	e	R	R	e	e	e	e	N/A	N/A
I11	APIP Accessibility	N/A	N/A	X	e	R	R	R	R	N/A	N/A
Accessibility
A1	Spoken*, Text Only	N/A	N/A	N/A	e	R	R	R	R	e	R
A2	Spoken, Text & Graphics	N/A	N/A	N/A	e	e	R	X	R	e	R
A3	Spoken, Non-Visual	N/A	N/A	N/A	e	e	R	X	R	e	R
A4	Spoken, Graphics Only	N/A	N/A	N/A	e	e	e	X	e	e	e
A5	Spoken, Directions Only	N/A	N/A	N/A	e	e	e	X	e	e	e
A6	Spoken, user preferences, including recording speed (slow, medium, fast) and voice type (human or synthetic)	N/A	N/A	N/A	e	e	e	X	e	e	e
A7	Screen Reader preferences (usage, speech rate, pitch, etc., for synthetic text-to-speech)	N/A	N/A	N/A	e	e	e	X	e	e	e
A8	Braille	N/A	N/A	N/A	e**	e**	e**	X	e**	e	R
A9	Braille user preferences (grade, dots, cells, etc.)	N/A	N/A	N/A	e	e	e	X	e	e	e
A10	Tactile	N/A	N/A	N/A	e	e	e	X	e	e	R
A11	Signing: ASL	N/A	N/A	N/A	e	e	e	X	e	e	e
A12	Signing: Signed English	N/A	N/A	N/A	e	e	e	X	e	e	e
A13	Item Translation	N/A	N/A	N/A	e	e	e	X	e	e	e
A14	Keyword Translation	N/A	N/A	N/A	e	e	e	X	e	e	e
A15	Simplified Language	N/A	N/A	N/A	e	e	e	X	e	e	e
A16	Alternate Representation	N/A	N/A	N/A	e	e	e	X	e	e	e
B1	Magnification	N/A	N/A	N/A	e	R	R	R	R	R	R
B2	Magnification, Amount (user preference)	N/A	N/A	N/A	e	e	e	X	e	e	e
B3	Reverse Contrast	N/A	N/A	N/A	e	e	e	X	e	e	e
B4	Alternate Text and Background Colors	N/A	N/A	N/A	e	R	R	R	R	R	R
B5	Alternate Text and Background Colors, Color Preferences	N/A	N/A	N/A	e	e	e	X	e	e	e
B6	Color Overlay	N/A	N/A	N/A	e	e	e	X	e	e	e
B7	Color Overlay, Color Preferences	N/A	N/A	N/A	e	e	e	X	e	e	e
C1	Answer Masking	N/A	N/A	N/A	e	e	e	X	e	e	e
C2	Masking	N/A	N/A	N/A	e	e	e	X	e	e	e
C3	Auditory Calming (background music)	N/A	N/A	N/A	e	e	e	X	e	e	e
C4	Additional Testing Time (including time & additional testing time tags)	N/A	N/A	N/A	e	e	e	X	e	e	e
C5	Breaks	N/A	N/A	N/A	e	e	e	X	e	e	e
C6	Keyword Emphasis	N/A	N/A	N/A	e	e	e	X	e	e	e
C7	Line Reader	N/A	N/A	N/A	e	e	e	X	e	e	e
C8	Line Reader, Color Preferences	N/A	N/A	N/A	e	e	e	X	e	e	e
C9	Language Learner Guidance	N/A	N/A	N/A	e	e	e	X	e	e	e
C10	Cognitive Guidance	N/A	N/A	N/A	e	e	e	X	e	e	e
D1	Calculator Companion Material	N/A	N/A	N/A	e	e	e	X	e	N/A	N/A
D2	Rule Companion Material	N/A	N/A	N/A	e	e	e	X	e	N/A	N/A
D3	Protractor Companion Material	N/A	N/A	N/A	e	e	e	X	e	N/A	N/A
D4	Reading Passage Companion Material	N/A	N/A	N/A	e	e	e	X	e	N/A	N/A
D5	Digital Companion Material	N/A	N/A	N/A	e	e	e	X	e	N/A	N/A
D6	Phyiscal Companion Material	N/A	N/A	N/A	e	e	e	X	e	N/A	N/A
Results
R1	Context	e	N/A	X	e	N/A	N/A	N/A	N/A	N/A	N/A
R2	Test result	e	N/A	X	e	N/A	N/A	N/A	N/A	N/A	N/A
R3	Item Result	e	N/A	X	e	N/A	N/A	N/A	N/A	N/A	N/A
QTI Metadata
M1	Item Template	e	e	R	R	e	e	e	e	N/A	N/A
M2	Time Dependent	e	e	R	R	e	e	e	e	N/A	N/A
M3	Composite	e	e	R	R	e	e	e	e	N/A	N/A
M4	Interaction Type	e	e	R	R	e	e	e	e	N/A	N/A
M5	Feedback Type	e	e	R	R	e	e	e	e	N/A	N/A
M6	Solutions Available	e	e	R	R	e	e	e	e	N/A	N/A
M7	Tool Name	e	e	X	e	e	e	e	e	N/A	N/A
M8	Tool Version	e	e	X	e	e	e	e	e	N/A	N/A
M9	Tool Vendor	e	e	X	e	e	e	e	e	N/A	N/A
Packaging
P1	Test Instances	e	R	R	R	R	R	R	R	N/A	N/A
P2	Section Instances	e	R	R	R	R	R	R	R	N/A	N/A
P3	Item Banks	e	e	R	R	e	R	e	R	N/A	N/A
P4	Item Instances	e	R	R	R	R	R	R	R	N/A	N/A
P5	Manifest Metadata (LOM)	e	R	R	R	R	R	R	R	N/A	N/A
P6	Resource Metadata (LOM)	e	R	R	R	R	R	R	R	N/A	N/A
P7	Resource Metadata (QTI)	e	e	R	R	e	e	e	e	N/A	N/A
P8	Curriculum Standards Metadata	e	e	e	e	e	e	e	e	N/A	N/A
P9	External Metadata	e	e	e	e	e	e	e	e	N/A	N/A
P10	Learning Tools Interop v1.1	e	e	e	e	e	e	e	e	N/A	N/A

 

Elective access features are those features that could be supported by systems, but are not required for conformance. Elective features will be individually certified, and systems that use elective features must list the specific APIP Elective access features they support. Either Entry or Core certified systems are eligible to be certified in any Elective feature. There is no Elective certification without Entry or Core certification.

The certification categories include a differentiation between the ability to import and export APIP Content or PNP files. Since APIP is primarily a transfer format, it will not certify how systems manage or modify data within the system itself. Delivery Systems are expected to combine the information provided by APIP content packages and PNP files. Although they do not need to natively use the APIP format during delivery, certified delivery systems must make use of the data/information supplied in APIP format.

Systems that import APIP information must distinguish between valid APIP formats and invalid formats, and provide information about which packages are invalid. These importing systems must support this format validation functionality for APIP Entry access features and may support additional APIP Core and Elective access features.

Systems that export APIP content or PNP instances must be able to supply valid APIP formatted instances, minimally supporting APIP Entry access features, and may support additional APIP Core and APIP Elective access features.

 

4                  Achieving Certification

Validation for APIP conformance will consist of the following strategies:

 

• Automated Validation. An electronic process to verify conformance using automated tools provided by the 1EdTech APMG, and
• Test Case Based Validation. Using a standard set of APIP/QTI content packages and PNP files with known characteristics provided by the 1EdTech APMG that must be used as part of the validation process and have known expected results that can be documented and/or demonstrated, and
• Documented or Demonstrable Validation. Documented statements and/or demonstrable capabilities showing conformance that can be reviewed by the 1EdTech APMG or by the user.

Because the APIP standard is primarily a data transfer format, conformance and certification does not govern how applications manage or modify data within the system itself. For example, delivery systems must be able to combine the information provided by imported content and PNP files, but need not natively use the APIP format during delivery. There are limits to what can be verified in an automated fashion. In the categories listed in the prior section as an example, it makes a clear distinction between import and export. The reason for this is that export functions are much more easily verified using automation. The output from an export process must adhere to the APIP XML specifications and can be verified using automation. Import, on the other hand, is much more difficult to verify because the import process results in a proprietary system consuming the information and representing it in an internal format that does not need to adhere to the XML specification and cannot be verified using automation.  Each of these approaches is discussed in detail below.

4.1            Automated Validation

Capabilities to be validated using automated tools include:

•         APIP Content Package export

•         APIP PNP file export

For the export, the APIPv1 conformance approach focuses on determining if the system can generate an object (PNP instance or item) that is non-problematic. 1EdTech provides an online tool available for individuals or organization to submit an object (PNP file, item) and then receive information about the existence (and sometimes nature) of problems with syntax and semantics of the object.

It is expected that the automated validation tool will:

1)      Accept Entry, Core, and Elective APIP conformance content elements identified in the conformance matrix.

2)      Accept custom (and profiled) extensions that are agreed to between exchanging parties (interdependent vendors).

3)      Identify which Core and Elective access features as well as the QTI v2.x features are implemented in the supplied instance.

4.2            Test Case Based Validation

Capabilities to be validated using a standard set of test cases include:

•         APIP Content Package import

•         APIP PNP file import

The APIP Alliance makes available sets of objects (PNP files, items) - some that are problematic and others that are non-problematic - for conducting import validation. Import functionality focuses on determining if the system can (a) flag as problematic objects that are known to have problems in syntax or semantics and (b) recognize as non-problematic the objects that are known to be without problems.

4.3            Documented or Demonstrable Validation

Capabilities to be documented and/or demonstrated by the solution provider include:

• Render APIP content, supporting differences in PNP file information

As each test delivery system may provide a unique user experience and use different technologies to deliver to content, providing automated validation of each implementation is virtually impossible. Therefore, the 1EdTech APMG will rely on the solution provider to document that the required APIP content and PNP features have been implemented and can document and/or demonstrate them. The documents and demo systems can be submitted to the APMG for consideration and review but the APMG will not be able verify the exact system behaviors.

 

Appendix A - Accessibility Features

 Online accessibility may be thought of as consisting of four major categories for supporting accessibility:

A.    Accessibility through Alternate Representations

B.     Accessibility through Adapted Presentations

C.     Accessibility through Adapted Interactions

D.    Accessibility through Adapted Response Modes

Future versions of the APIP standard may (1) include access features for category D (Adapted Response Modes), (2) add new access features to approaches A, B, and C, and (3) change the core/elective designation of a feature (e.g., for the delivery area, change Braille from elective to core).

The complete list of APIP v1 access features are listed below, organized by stated categories above.

A: Alternate Representations

Spoken Support (sometimes called read aloud or verbal): Text presented to the user is spoken aloud. Graphics (tables/diagrams/pictures) would have alternate text that could be spoken aloud.

A1    Spoken, Text Only: Only text (alpha-numeric) content will have spoken support. At the discretion of the author, some text may have enhanced spoken information, or may not be included (like reading numbers in a math assessment). Graphics would NOT be described to this user.

A2    Spoken, Text and Graphics: Text content as well as graphic content will be described to the user. Some more contextual information about layout or object orientation may be added for this user, depending on the content.

A3    Spoken, Non-Visual: All content is read and described to the user, as the expectation is that the user cannot see the content. Contextual information about layout or object orientation may be needed for some content. Special consideration may be needed for the order in which the information is presented.

A4    Spoken, Graphics Only: Only graphic descriptions would be read aloud to this test taker. Test takers access this information on-demand only (i.e., at the test taker's request).

A5    Spoken, Directions Only: Only test directions are read aloud to the user.

A6    Recording Speed (Slow, Medium, Fast): the rate of speech that a recorded audio file presents spoken verbal information. Systems that can adjust the rate of speech for pre-recorded audio files at the moment of delivery might adjust the rate based on the Medium (or regular) rate of speech.

         Voice Type (Human or Synthetic): whether a pre-recorded audio file was created by a human reading text, or generated by computer speech emulation (Synthetic).

               For Content compliance, content would provide specific information about the sound files provided within the content, allowing user preferences about the type of audio representation to be considered for use during presentation. For PNP compliance, the system would store the user's spoken preferences. For Delivery compliance, the system would provide the user's spoken preferences (using their Personal Needs and Preferences information) to the delivery system when the testing session is initiated.

A7    Screen Reader Preferences (usage, speech rate, pitch, etc.): preferred characteristics of computer generated voice for the test taker. For PNP compliance, the system would store the user's synthetic spoken preferences. For Delivery compliance, the system provides specific synthetic spoken preferences (using their PNP information) to the delivery system when the testing session is initiated.

Braille: Content would have specific text strings to be used in a refreshable Braille display device. The device is expected to convert the text into Braille.

A8    Braille: All content is presented as Braille characters and described to the user, as the expectation is that the user cannot see the content. Contextual information about layout or object orientation may be needed for some item or test content.

A9    Braille user preferences (grade, dots, cells, etc.): Allows the system to provide the user's preferred Braille preferences (using their Personal Needs and Preferences information) to the delivery system for use with a refreshable Braille display.

Tactile: An external, physical representation of parts of the item content, or the item's companion materials.

A10  Tactile: A tactile representation of the graphic information is made available outside of the computer testing system. The access element tags should include descriptions of how to locate the specific tactile sheet (using referencing numbers/letters/symbols) needed to respond to the item. This support is simply a reference to tactile materials for the test taker. It does not provide a description of how to form/create the physical, tactile representation.

Sign Language: Animated or live-action sign language movie recordings are provided to the user.

A11  Signing Type: ASL: Additional American Sign Language content is provided to the user. Unlike other translated content, ASL content is presented in addition (or in parallel) to the default content.

A12  Signing Type: Signed English: Additional Signed English content is provided to the user. Unlike other translated content, Signed English is presented in addition (or in parallel) to the default content.

Item Translation: A different language variant of the same item.

A13  Item Translation: An alternate item is made, and the user is exposed to the alternate language version. The content package would contain which specific language it is providing.

Keyword Translation: parts of item or test content are translated and available to specific test takers.

A14  Keyword Translation: Certain specific words or phrases would have translations available to users who need some assistance with difficult or important words in the content. The test taker profile would specify the language requested, and the content would supply the translation for the program-required languages.

Simplified Language: An entirely different variant of the item is given to the user, using simpler language.

A15  Simplified Language: the test taker is presented with the Simplified Language variant of the item (if available). Not at items in a particular assessment may need simplification. The default content may be considered accessible by test taker's requiring Simplified Language support.

Alternate Representation: Specifies an alternate way of displaying content to facilitate the user's access to the content. As an example, a text-based description of a figure displaying a life cycle might be provided, or an animation that represents a series of events described in text might be provided.

A16  Alternate Representation: present a text string alternative of the original representation.

 

B: Adapted Presentations

Magnification: All content is magnified by the amount specified by the user.

B1    Magnification: Provides the capability to magnify content.

B2    Magnification, Amount: For PNP compliance, the system stores the user's preferred magnification amount. For Delivery compliance, the system provides the user's preferred magnification amount (using the PNP information) when the testing session is initiated.

Color Adjustments: specific changes are made to the interface at the request of the user that allows them to visually access the content or interface.

B3    Reverse Contrast: inverts all the color values in the assessment environment. Black is white, white is black, red is green, blue is orange, etc.

B4    Alternate Text and Background Colors: The interface allows the user to select alternate colors for the text (foreground) and background colors.

B5    Color Preferences for Alternate Text and Background Colors: the stored, specific color preferences of the test taker. For PNP compliance, the system stores the user's color preferences for the text and background.  For Delivery compliance, the system provides the user's preferred color choices (using their PNP information) to the assessment delivery system when the testing session is initiated.

B6    Color Overlay: A color tint is layered over the content to aid in reading the item and test content. It emulates the classroom practice of using color Acetate (transparent plastic) sheets over paper.

B7    Color Preferences for Color Overlay: the stored, specific color preferences of the test taker. For PNP compliance, the system stores the user's color preferences.  For Delivery compliance, the system provides the user's preferred color choice (using their PNP information) for the overlay to the assessment delivery system when the testing session is initiated.

 

C: Adapted Interactions

Masking: Portions of the content and interface may be covered to reduce the amount of stimulus provided to the student during testing.

C1    Answer Masking: By default, answer choices for multiple-choice items are covered when the item is first presented. The user has the ability to remove the masks at a time of their choosing.

C2    Masking: The user is able to create their own masks to cover portions of the item until needed.

Auditory Calming: The test taker can listen to music or sounds in the background as they respond to questions during the testing session.

C3    Auditory Calming: Background music or sounds

Additional Testing Time: If a test has a time limit, the user will be allowed additional time (specific additional amounts or indefinite) to complete the test.

C4    Additional Testing Time: time and additional testing time are specified by the test taker's user profile. This support is provided without the ability for the test taker to alter or remove the support.

Breaks: User is allowed to take breaks, at their request, during the testing session.

C5    Breaks

Keyword Emphasis: Certain words are designated in the content as key words. Contracting programs would designate how they are to be emphasized (bold, italic, etc.)

C6    Keyword Emphasis

Line Reader: Test taker has an on-screen tool available that assists them in moving a reading tool (line highlighter or underscore) down (or up) the block of text, line by line, to assist in reading the content.

C7    Line Reader: Test taker has access to a Line Reader Tool.

C8    Color Preferences for Line Reader: the stored, specific color preferences of the test taker. For PNP compliance, the system stores the user's color preferences.  For Delivery compliance, the system provides the user's preferred color choice (using their PNP information) for the color overlay in the Line Reader Tool to the assessment delivery system when the testing session is initiated. Note: not all Line Reader Tools use color to help highlight or guide reading.

Guidance: The test taker is provided with supplemental information about all or portions of the item/test content to aid in their processing of the presented information.

C9    Language Learner: Additional information is provided in the test language about words or phrases that are intended to assist the language learner process those words or phrases.

C10  Cognitive Guidance: Additional information is provided to assist a test taker process or understand all or part of an item. Guidance may include how they are expected to respond to an item (e.g., additional instructions that emphasize that the student should use the information presented in a graph to answer the question).

 

D: Companion Materials

D1    Calculator: All calculators would have access to all number digits, decimal key, equals button, and clear button. Read aloud capability should be something that is configurable to either allow or not allow during testing. Additionally, some institutions allow for reading the numbers or functions as you use them, but do not allow reading the number as a whole. This is usually for Math related content. The four possible calculators that can be specified are Basic, Standard, Scientific, and Graphing.

D1a        Basic Calculator: Assumed functions: Add, Subtract, Multiply, Divide.

D1b        Standard Calculator: Assumed functions: all basic calculator functions, Square root, Percentage (%) , Plus/Minus, a.k.a. Sign Change, Memory Functions

D1c        Scientific Calculator: Assumed functions may include, but are not limited to: all standard calculator functions, a π key, sign change (+/-), square (x²) , cube (x³),  x to the y (x^y), square root (√) , cube root , xth root , logarithm keys, log, ln, base 10, base e, Trigonometry function keys with an INVERSE key for the inverse functions, sin, cos, tan, hsin (hyperbolic sin), hcos, (hyperbolic cos), htan (hyperbolic tan), DEG, RAD, GRAD conversion, a capacity to work in both degree and radian mode, a reciprocal key (1/x) - calculate the inverse of the displayed value, permutation and/or combination keys (nPr , nCr), parentheses keys, metric conversion, permutation and combination keys, nPr,  cPr, x!

D1d        Graphing Calculator: A Graphing calculator includes many of the same functions of a scientific calculator, plus the ability to display equations graphically.

D2    Rule: Allows for the presentation of a measuring device for use on the computer with the supplied content. The ruling system can be specified as metric (SI) or US measurement systems.

D3    Protractor: The examinee will be supplied with an on-screen protractor while responding to the item. Increments can be specified as metric (radians) or US (degrees) systems of angular measurement.

D4    Reading Passage: Allows a reference to a reading passage that needs to be provided to the test taker while responding to an item.

D5    Digital Materials: Content or reference materials that relate to the item content. Examples could be a map, a table of information, a sheet of math formulas, an interactive periodic table of elements, or even graphic creation tools.

D6    Physical Materials: External materials needed to work with, or respond with, when the test taker responds to the item.

Appendix B - RFP Language for APIP

This appendix provides guidance to organizations or programs that wish to require APIP in an RFP that they issue.

A successful accessible assessment program requires the coordination of assessment content, learner preferences and the supporting assessment delivery technology platforms. When considering how to specify the requirements for your programs, all three must be considered together. When choosing specific accessibility features for your assessment program, you will need to consider how each feature will impact these three areas.

Every assessment program is unique. What requirements that you identify for your program will take precedence over any requirements provided in this sample text. APIP compliance, as defined by 1EdTech, pertains to the standard implementation itself. The contract you have with your service provider will ultimately dictate the obligations your service provider must achieve.

Close consideration should also be given to paper-based, online, and mixed mode assessment programs and how each may impact accessibility requirements. Some accessibility options, required or elective, will have cost implications for the development and implementation of content and processing systems. Special consideration should be given for programs that plan to repurpose existing content repositories that currently do not provide the necessary accessibility information. In addition, an existing repository may not be formatted in the QTI v2.1 format. You may wish to provide pricing options so that your vendors can identify those items separately.

Note: For text below that is enclosed in square brackets [ ], it is assumed that the requesting organization will modify or insert text as appropriate for your program.

The suggested RFP language follows.

B1 - Accessible Portable Item Protocol Standard

The Accessible Portable Item Protocol (APIP) Standard is a freely available specification for standardizing the exchange file format of digital test items for enabling accessible assessments. By extending the QTI v2.1 standard, APIP allows accessibility information to be included in a digital test item definition so that items can be transferred between APIP compliant item banks.

When combined with an individual learner's accessibility needs and preferences, an assessment delivery system can implement interfaces required to make assessments accessible to learners with a variety of disabilities and special needs. APIP offers a tagging map to the 1EdTech Personal Needs and Preferences (PNP) standard that can seamlessly link a learner's accessibility information to assessment content.

In summary, the response must describe how the assessment solution will approach the following [please remove any that are not included in this RFP/RFI/other]:

• Importing and exporting APIP formatted item and test content into and from the host item banks (there is no requirement for storing items and test content in APIP format within the host item bank),
• Providing an authoring environment that allows for the review and editing (adding, updating, or removal) of accessibility information for items and tests,
• Providing an authoring environment that allows the items to be structured with the appropriate test and form definitions and associated meta-data and retain all required accessibility information for the target delivery system(s),
• Integrating the learners accessibility needs and preferences (such as the 1EdTech PNP) with the assessment registration process,
• Publishing optimized and platform specific test content, including items imported from APIP sources, to enable the necessary accessibility features and functions in the target technology and delivery environment,
• Providing administrator/proctor controls to extend time or suspend sessions for breaks based on an individual learner's needs,
• Providing an assessment delivery platform that can marry the individual learner's needs and preferences with the accessible content and deliver the desired features and functionality to each test taker.

The APIP Conformance documentation provides for a table of QTI, PNP, and APIP elements (tags) with a designation of the element being an Entry level or a Core level element. The Entry level indicates those elements that are expected, at a minimum, to be in place for any APIP compliant application. The Core level entries provide for 'implementation electives' that can be provided by the application or can be added as required elements for any specific program. The conformance documentation can be found at http://www.imsglobal.org/apip.

[The requesting organization should review the list of elements in the conformance documentation and note any program specific variations that may apply to your program. This may include removing unnecessary, unwanted, or not-applicable features for your program or designating elective features as required for your program.]

The response to the RFP should include the conformance matrix from this APIP Conformance document with a description of how each feature (excluding any features noted above) is addressed by the solution being proposed.

B2 - Compliant Content Format

The APIP item standard includes provisions for accessibility information that extends the default representation of item content (QTI v2.1) to include a large variety of access features that support the default representation.

The content should support specific inclusion orders and associated access elements in the content for the following accessibility categories [remove any that do not apply or note as optional].

• Spoken: Text Only
• Spoken: Text & Graphics
• Spoken: Non-Visual
• Spoken: Graphics only
• Braille
• Signing: ASL
• Signing: Signed English

The following list identifies areas where full accessibility compliance may be excluded by an issuer of an RFP. The responder should articulate how such content will be identified in each category [remove any that do not apply or note as optional; note: while contract exceptions may be allowed by an issuer of an RFP, compliance for systems or instances will not be permitted in the certification process].

• Items that contain novel interfaces or content elements that are not represented within the APIP or QTI Item Standards.
• Item content developed under prior test administrations or contracts that may be difficult or costly to adapt to meet all required elements of the APIP standard.
• Items not intended for delivery using accessible interfaces such as Braille or online accessibility interfaces.
• Computer adaptive administration strategies for excluding items from selection where items do not include all the necessary accessibility elements required for the learner being tested.

B3 - Learner Information Profiles

The assessment administration system must be able to accept learner information files containing the required accessibility components as defined in the matrices above. The [state/districts] will provide the learner's accessibility profile information in the following ways [remove any that do not apply or note as optional]:

• The current registration process and data files will be extended to include the required profile elements as identified in the matrices above.
• As a separate data collection using PNP standard data specification and format (XML).
• As a separate data collection using a separate file specification.
• Using the online management features of the administration system to enter or update personal profile data elements directly in the assessment application.
• Using other industry standards that specify comparable profile information.

The [vendor or state] will provide instructions and additional resources to the local districts and schools needing to generate the necessary profile data or files as part of the assessment registration or supplemental process.

B4 - Digital Test Delivery System

The digital test delivery system employed to deliver test items to learners must have all the required features and functions for those elements identified in this [RFP/RFI/other]. The test delivery system must be able to present the specified accessibility features based on the learner's preferences. Specifically, the test delivery system must be able to provide the accessibility features as defined in the tables above. In addition, the test delivery system must allow learners to interact with the system using a standard compliant point device (mouse), keyboard (standard and alternate), touch screen, and/or a tab-enter enabled device. If the delivery system requires additional hardware components or is incompatible with specific devices, the responder should specify those requirements in the response. Responders must identify the supported technology delivery platforms (hardware requirements and software with versions) and any limitations or considerations that must be given for accessibility features.

[Note 1: You may wish to also state expectations. Possible examples are one or more of the functional performance criteria of the section 508 standards (subpart C; http://www.access-board.gov/sec508/standards.htm), which might be adapted to an assessment delivery setting.]

[Note 2: The APIPv1 braille feature provides text that can be sent to a refreshable braille display. It is required for Core level authoring and PNP systems. For all other conformance levels (e.g., Entry) and systems (delivery), any requirement for refreshable braille must be explicitly stated. Furthermore, because support for hard copy braille is outside of APIPv1, any requirement for hard copy braille must also be made explicit.]

B5 - APIP Conformance Certification and Evolution

The 1EdTech Consortium has established a conformance testing and certification program for APIP. Conformance testing and certifications are available for content import and export from assessment item banks. In addition, 1EdTech manages the ongoing evolution of the underlying specifications, most notably Question and Test Interoperability (QTI) and the Personal Needs and Preferences (PNP).

All supported versions of QTI and APIP should be identified. Plans should also be defined that will address the processes, costs, and timelines for upgrading assessment content, learner profiles, and assessment technology platforms as newer version of QTI and APIP are released to the public.

About this Document

Title:                                       1EdTech APIP Conformance and Certification

Editors:                                  Colin Smythe (1EdTech) and Mark McKell (1EdTech)

Co-chairs:                              Gary Driscoll (ETS), Thomas Hoffmann (Measured Progress), and Wayne Ostler (Pearson)

Version:                                  1.0

Version Date:                         31 March 2014

Status:                                    Final Specification

Summary:                              The aim of the APIP Project is to use well established e-learning interoperability standards to enable the exchange of accessible assessment content between computer-based assessment systems, tools and applications.  Users of systems, tools and applications that adopt the APIP are able to use their accessible assessment content on a wide range of systems.

Purpose:                                 This document is made available for adoption by the public community at large.

Document Location:             http://www.imsglobal.org/apip/

 

List of Contributors

The following individuals contributed to the development of this document:

Rob Abel, 1EdTech (USA)	Justin Marks, NWEA (USA)
Michael Aumock, Pacific Metrics (USA)	Mark McKell, 1EdTech (USA)
Marty Christensen, ACT (USA)	Sue Milne, JISC (UK)
Jason Craft, Pearson (USA)	Wayne Ostler, Pearson (USA)
Gary Driscoll, ETS (USA)	Zack Pierce, Measured Progress (USA)
Eric Hansen, ETS (USA)	Michelle Richard, Pearson (USA)
Regina Hoag, ETS (USA)	Mike Russell, Measured Progress (USA)
Thomas Hoffmann, Measured Progress (USA)	Farhat Siddiqui, ETS (USA)
Wilbert Kraan, JISC (UK)	Colin Smythe, 1EdTech (UK)
Devin Loftis, McGraw-Hill/CTB (USA)	Wyatt VanderStucken, ETS (USA)

 

Revision History

 

Version No.	Release Date	Comments
Candidate Final v1.0	26 March 2012	The first formal release of the Candidate Final version of this document.
Final Specification v1.0	31 March 2014	The Final Specification version of this document.

 

 

 

1EdTech Consortium, Inc. ("1EdTech") is publishing the information contained in this document ("Specification") for purposes of scientific, experimental, and scholarly collaboration only.

1EdTech makes no warranty or representation regarding the accuracy or completeness of the Specification.

This material is provided on an "As Is" and "As Available" basis.

The Specification is at all times subject to change and revision without notice. 

It is your sole responsibility to evaluate the usefulness, accuracy, and completeness of the Specification as it relates to you.

1EdTech would appreciate receiving your comments and suggestions.

Please contact 1EdTech through our website at http://www.imsglobal.org.

Please refer to Document Name: 1EdTech APIP Conformance and Certification v1.0 Final Specification

Date: 31 March 2014