Q. Got a question?
Submit one to the APIP public forum here: http://www.imsglobal.org/community/forum/categories.cfm?catid=110 and we will update the FAQ with new questions and answers.
Q. Why is there a need for portability of assessments?
Standards exist for many applications and types of information stored and transferred between computers, a common method for coding, storing, transferring and presenting computer-based test items is also imperative for the K-12 assessment industry. Vendors often rely on their own proprietary methods for coding test items, using Hypertext Markup Language (HTML), Extensible Markup Language (XML), Flash, or other non-standard methods. The need for a common way of coding computer-based items is imperative so assessment items and tests can be viewed, repurposed, and transferred between all test delivery systems and item banks.
Q: Why is there a need for accessibility in assessments?
Assessment at all levels—classroom, school, district, and state—is essential for tracking student success, guiding instruction, and for learning how we can improve our school systems. This has been demonstrated by extensive research in the U.S. and abroad. However, under state and federal laws, all students must be able to perceive, interact with, and respond to the full range of assessments in order for us to obtain valid and meaningful information about learning that teachers and parents need and can use to foster student growth. When a student can perceive, interact with, and respond to instruction and assessment we say that student has achieved meaningful “access” to education. This means that it is possible to use assessments to help us collect valid information about what that student really knows, understands, and can do. Conversely, if a student cannot access instruction or assessment, it is not possible for us to gain meaningful information about what the student knows or might be able to learn. Without meaningful access, we have no way to inform or guide instruction because we have no way to communicate effectively with the student regarding his or her understanding of academic concepts.
Unfortunately, an “assessment gap” exists for far too many students. They can’t fully access test materials to accurately demonstrate what they know and can do. This sad fact applies to state tests, which typically offer the most accommodations and supports, and even more so to district, school, and classroom assessments, which typically lack accessibility supports. As a result, these students don’t receive the instructional support they need.
Q. What role does U.S. federal legislation play in accessibility?
Over the last century, several major federal laws have been enacted for the purpose of protecting the rights of each and every citizen, regardless of the presence of a disability, to have equal access to all publicly funded education, training, and employment preparation programs. Examples of these laws include: The Smith Fess Act (1920), The Rehabilitation Act of 1943 (reauthorized across the decades and now commonly known as Section 504 and Section 508 rights); The Elementary and Secondary Education Act of 1965 (still active); The Education of All Handicapped Children’s Act of 1975 (now known as IDEA); and The Americans with Disabilities Act, enacted 1990 (and still active). All these laws and many others have evolved across the decades as our understanding of the diverse and resilient nature of human capabilities has grown and deepened.
The Individuals with Disabilities Education Act of 1994 (IDEA ’94) raised the importance of providing all students with access to instructional materials. IDEA ’97 included a more exacting mandate: all students had to be tested for their achievement of state standards. This legislation drove the application of Universal Design principles to instructional materials and then assessments. As the publishing industry began to develop digital content, the National Instructional Materials Accessibility Standards (NIMAS) were developed and included in IDEA ‘04, a major step forward for many students.
Q. What efforts have been under taken to build accessibility standards for assessment content?
Recent technological advances and the growing importance of—and unique demands inherent in— assessment drove efforts to increase the accessibility of test content for all students, illustrated for example by the requirements for the U.S. Federal Department of Education’s (USED) Race to the Top Assessment Program. To develop an industry standard for accessibility and interoperability of test items, USED funded the Accessible Portable Item Protocol (APIP) project. The Minnesota Department of Education led the effort, which included the states: New Hampshire, Vermont, Utah, Montana, Florida, South Carolina, and Maryland as participants and Michigan, Massachusetts, and North Carolina as observers. National interoperability and accessibility experts provided technical support. In December 2010 the team released the first version of the APIP standard, intended to make assessment content portable between systems and accessible to a wide range of students.
Q. So what exactly is APIP?
APIP is a highly efficient interoperability solution for authoring and delivering test items to millions of students – who receive these items through many different delivery platforms across the country or the world – but in a way that supports consistency of the delivery of each and every item. In fact, because of the efficiency of this common coding solution, we can now meet the educational needs of many more students than ever before possible.
Built on existing standards that service providers can readily adopt, the APIP standard provides the technical framework required to modify or develop item authoring, item banking and test delivery systems so they can create and deliver items and tests that are—for the first time in our field —portable and highly accessible. As illustrated in this figure, the specifications within APIP cover three critical areas:
Q. Who maintains APIP?
APIP is maintained by IMS Global Learning Consortium (http://www.imsglobal.org/) as a freely available, open-licensed standard. IMS is a non-profit, member organization that supports the growth and impact of learning technology worldwide. IMS’s APIP Workgroup includes several state and testing industry members, including Measured Progress, Pearson, ETS, CTB-McGraw Hill, ACT, Pacific Metrics.
Q. What standards is APIP based on?
APIP is based on three existing interoperability standards:
Q. What does it mean to be APIP conformant?
APIP Conformance describes two tiers of accessibility features within APIP, namely Required (R) and Elective (e) features. Required accessibility features are features that implementing systems are expected to support as a base level, or minimal feature set, of an APIP certified system. This minimal feature set is called the Entry APIP “conformance profile”. At present there are two officially recognized conformance certification levels: Entry Level and Core Level. The Core Level adds additional accessibility features that must be supported by implementing systems. In addition to accessibility features, there will be requirements set for the minimum QTI features (as defined within the APIP QTI 2.1 profile) for Content and Delivery Systems. QTI features include assessment interoperability features such as item types, scoring, and feedback features. Depending upon which feature set is demonstrated by the implementing system, the system will receive either APIPv1 Entry or Core Certification. The specific required features for each level of certification are outlined in the table below. Elective access features are those additional features beyond the Entry or Core sets that would be optionally supported by systems, at the request of customers. These elective features would be individually certified, and certified systems that include them will list the specific APIP elective access features they support, using the conformance identifiers defined within the conformance documentation.
The IMS conformance process allows for additional profiles to be created based on regional or community-specific requirements. For example, the RttTA consortia (PARCC, Smarter Balanced) may determine that the APIP Core or Entry profiles do not fully meet their community’s specific requirements and a new conformance profile could be developed to which vendors would develop content and systems for certification.
Q. What does it mean to be APIP certified?
IMS stands behind the conformance marks it issues to products and content that prove conformance to the standard. When APIP systems and content are ready to prove conformance, they will be listed at www.imscert.org and will receive a Conformance Registration Number and be able to display the official certified logo that will ensure that the specific version of an available product is IMS certified. IMS only certifies specific named products and specific versions of those products. IMS certification adds value to the marketplace because it means that the supplier is committed to working with the IMS community to actively resolve issues that may arise.
This www.imscert.org web page is the only official listing of products that have received IMS conformance marks. Many more suppliers around the world use IMS standards, but achieving the conformance mark indicates that a product has gone through and passed testing prescribed by the IMS members in an ongoing community process.
Q: Does an ‘R’ in an authoring product mean that all items must have that accommodation or accessibility tool?
If an item is expected to be delivered to an audience requiring the accessibility content, then the Content Authoring System should be providing the accessibility content within the item. If an item is specifically deemed inappropriate for a specific audience, and the item will not be delivered to that audience, then those exceptional pieces of content do not need the accessibility information. For example, if you have created an item that you know would be impossible for a blind person to respond to correctly, that item would not include Braille content, a Braille inclusion order, nor would it include a Spoken NonVisual inclusion order. It should be noted too that IMS conformance is relevant to the technical inclusion of the information, not the accuracy or appropriateness of the information.
Q: Does an ‘e’ mean it won't be implemented at all or just not tested or just not part of certification?
Elective access features are those features that could be supported by systems, but are not required for conformance. Elective features will be individually certified, and systems that use elective features must list the specific APIP Elective access features they support. Either Entry or Core certified systems are eligible to be certified in any elective feature. There is no Elective certification without Entry or Core certification. If a vendor is contracted to deliver or receive Elective information, that vendor would be “required by contract” to support the Elective feature. The contract may or may not require that the vendor seek IMS certification for the specific Elective features specified in that contract.
Q: Why is this the list of accommodations supported? Can there be others?
The list of accessibility features in the current version of APIP were created by an initial end user group made up of 10 U.S. states, and then discussed and ratified by the APIP Workgroup, a technical working group made up of representatives from a number of different assessment vendors. These features were deemed to be vital to assessment accessibility and interoperability, and the vendor community believes they can reasonably support the current version’s accessibility features. Other features can be added in future versions of APIP by bringing the feature requirements to either the APIP Workgroup or APIP End User Group’s attention.
Q: Does conformance to APIP enforce how a delivery system presents assessments to students?
The conformance categories include a differentiation between the ability to import and export APIP Content or PNP files. Since APIP is primarily a transfer format, it will not certify how systems manage or modify data within the system itself. Delivery Systems are expected to combine the information provided by APIP content packages and PNP files. Although they do not need to natively use the APIP format during delivery, certified delivery systems must make use of the data/information supplied by APIP. Contracts with vendors who are providing compliant APIP Delivery Systems may include additional delivery specific requirements, including specifications around the presentation of default content, or how certain accessibility information should be presented. Bear in mind that some specific delivery-system implementation features may be the intellectual property of specific vendors, and may not be universally available. For example, a vendor may have developed a specific software tool for magnifying the content and navigating through that magnified content. The APIP concept of magnification is a required feature that compliant delivery systems must support, but a specific vendor’s implementation of that feature may be limited to that specific vendor.
Q. What is an Inclusion Order?
One important aspect of APIP is the concept of Inclusion Orders. With APIP, you can specify the order that information is supplied to different kinds of audiences. So, content can be read differently depending on the type of read aloud support (“spoken” in APIP terms) required by the student. Specifically, only text might be read for some users, only descriptions of graphics provided for other users, both text and graphics read for still other users, and more detailed descriptions of graphics provided for uses who have visual needs. For each category of user, both the information and the order in which information is presented to them are specified by an inclusion order. Think of inclusion orders as a specific audience’s content presentation order.
Also, for the different kinds of users, you can specify which content could be read to them automatically, and which content can be read at the request of the user. APIP refers to those as the Default Order and the On Demand Order.
Q: What are the differences between APIP and QTI?
QTI v2.1 and APIP are the same excepting that APIP includes support for a range of "accommodations" or access tools needed for students with disabilities taking assessments in an electronic setting - see the "Access Features" section in the table above. APIP is a higher bar to implement because it requires additional features in either an authoring tool (to author items, quizzes, tests) or test delivery platform to implement the accessibility features. APIP has been largely driven by RttTA and will be implemented by Assessment Systems.
If a vendor implements APIP then they are also implementing QTI v2.1. An APIP item or test that has accessibility features set to ‘none’ in an actual test or item is using QTI v2.1. If a vendor implements QTI v2.1 they are a long way towards getting to APIP, but are not there yet.
Q: Does APIP support of Braille include the Nemeth codes needed for symbolic/mathematical representation?
Version 1 of APIP does not support Nemeth codes because it doesn't actually support the attachment of BRF files to the content. We have proposed BRF files be attachable in version 1.1. This would allow the encoding using Nemeth code, though that becomes more of a policy decision really. The standard will support its use within the BRF files and strings (it’s just ASCII encoding).
Q: Some assessments use or might use “word prediction” – kind of like the spelling completion in Google or other web apps. Does APIP cover this?
APIP is silent on word prediction as part of interoperability requirements. It doesn't currently cover any aspect of student input. If the industry thinks it is important, a variable for the allowance or preference of using word prediction could easily be added to the APIP PNP specification.
Q: There are needs for different types of calculators, including a “spoken” calculator, is this an explicit type?
APIP doesn't specifically mention the accessibility features of the calculators, though it does differentiate between the functions a calculator should have. It is implied that the delivery engine would provide the accessibility supports required by students who need a spoken support. This requirement could be more explicit in future conformance documentation, if desired.
Q: Does APIP support increased whitespace?
Increased whitespace was originally part of the APIP specification, but has been profiled out because (at the time) states felt more research needed to be done on the effects of the various kinds of whitespace that could be added (line spacing, word spacing, letter spacing, and combinations of all three). If needed, and explicit requirements were provided, it could easily be added to an upcoming version.
Q: Can objects be tagged as 3-D and then not shown if a student doesn’t do well with 3-D objects? Does APIP support this?
We currently do not have a way of tagging certain content for a particular cognitive mismatch (where the content’s presentation does not match the capabilities of the user). Is that a desired feature? The workgroup could discuss the possible technical solutions to address this need and include it in a future version of APIP. It would likely involve all 3 aspects of APIP (content, PNP, and delivery).
Q: Can APIP separate delivery versus input requirements? That is, the student taking the test may not be able to use a computer and may need to provide input to another human being or other type of input?
No, APIP does not address this use case at this time. This could theoretically be easily solved through new variables added to the PNP specification, and included in version 1.1 or later.
Q: Does APIP support scaffolding?
Scaffolding was another feature that was in the original list of access features for APIP, and is currently profiled out of version 1.0. The states felt more research was needed before some standardized requirements could be given. If specific requirements were supplied, it could easily be added into a future version.
Q: Does APIP support graphic type alternate representations?
APIP version 1.0 does not support graphic alternate representations. Similar to previous questions about supported features, it is currently profiled out. More research was thought needed in order to determine what we mean by graphic representations. Did we mean specific graphical representations like bar charts, or line graphs, or is it just generically ‘graphic’? This could very easily be added to a future version.
Q: Does APIP support screen reader rate and pitch preferences for each individual?
Yes, APIP supports users providing their preferred reading rate and pitch. It is an elective compliance feature though, so would need to be specifically requested/contracted.
Q: Does APIP allow the ability to print out the stimuli, or print out an entire item?
In general, APIP leaves security details to the contracting parties, and is then left to the delivery system to address. If needed, a PNP variable could be added to indicate the student is allowed to print out testing materials. Another point is that APIP does allow you to reference physical materials that should be given to the user in conjunction with the item. Example might be a 3-D model or a book.
Q: Does APIP cover the use of companion materials (periodic table, formula sheet, etc.)?
Companion Materials are supported in APIP, and are described in section 2.2.13 on the Best Practices document. All companion materials can (should) themselves have accessibility information added to them (they can be APIP content).