1EdTech Caliper Analytics™ Implementation Guide
Version 1.0 Final
Date Issued: 15 October 2015
Latest version: http://www.imsglobal.org/caliper
IPR and Distribution Notices
Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the specification set forth in this document, and to provide supporting documentation.
1EdTech takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on 1EdTech’s procedures with respect to rights in 1EdTech specifications can be found at the 1EdTech Intellectual Property Rights web page: http://www.imsglobal.org/ipr/imsipr_policyFinal.pdf.
Copyright © 2015 1EdTech Consortium. All Rights Reserved.
Use of this specification to develop products or services is governed by the license with 1EdTech found on the 1EdTech website: http://www.imsglobal.org/license.html.
Permission is granted to all parties to use excerpts from this document as needed in producing requests for proposals.
The limited permissions granted above are perpetual and will not be revoked by 1EdTech or its successors or assigns.
THIS SPECIFICATION IS BEING OFFERED WITHOUT ANY WARRANTY WHATSOEVER, AND IN PARTICULAR, ANY WARRANTY OF NONINFRINGEMENT IS EXPRESSLY DISCLAIMED. ANY USE OF THIS SPECIFICATION SHALL BE MADE ENTIRELY AT THE IMPLEMENTER'S OWN RISK, AND NEITHER THE CONSORTIUM, NOR ANY OF ITS MEMBERS OR SUBMITTERS, SHALL HAVE ANY LIABILITY WHATSOEVER TO ANY IMPLEMENTER OR THIRD PARTY FOR ANY DAMAGES OF ANY NATURE WHATSOEVER, DIRECTLY OR INDIRECTLY, ARISING FROM THE USE OF THIS SPECIFICATION.
© 2015 1EdTech Consortium, Inc.
All Rights Reserved.
Trademark information: http://www.imsglobal.org/copyright.html
Document Name: 1EdTech Caliper Analytics Implementation Guide – Final v1.0
Revision: 15 October 2015
Executive Summary
The 1EdTech Caliper Analytics™ Learning Measurement Framework represents a valuable, standards oriented, high impact solution to a very significant problem amidst the online learning delivery ecosystem. Currently, this ecosystem has an insufficient, inconsistent, and fragmented underlying measurement and metrics capability across fast emerging of federated content/learning activity elements sourced from a wide array of providers. By providing the necessary alignment and structure to what is and should be measured, along with a framework to support the capture and marshaling of data, 1EdTech Caliper version 1.0 represents an achievable and valuable starting position. Moreover, the Caliper framework is extensible and aims to facilitate the likely iterative build-out, refinement, and enhancement of its capabilities.
Table of Contents
2.7 Assessment Item Metric Profile
2.10 Illustration of Metric Profile Interactions
3 Sensor Implementation Details
4.1 Transport-Only Scope for Sensor API and CaliperEvent
4.2 Caliper-Transport Event Sample Fixture
4.3 Caliper Transport and Conformance/Compliance Validation
5 Use Cases for Caliper Analytics
5.1 Caliper Enabled Engagement Sequence
5.3 Example Assessment Sequence
6.1 Caliper Conformance Testing Framework Diagram
6.2 Caliper Conformance Testing Details
7.1 Caliper Common Fixtures JSON
1 Introduction
The purpose of the 1EdTech Caliper project is to define a standard for enabling the collection of rich contextual data about learning interactions and a Sensor API™ for capturing and reporting this data. This work will enable learning environments to capture data from learning interactions and share it with other learning environments and consumers of learning analytics. This work will build on existing 1EdTech standards and culminate in a new 1EdTech standard.
The 1EdTech Caliper Framework is designed to meet the following requirements:
- Establish a means for consistently capturing and presenting measures of learning activity, which will enable more efficient development of learning analytics features in learning environments
- Define a common language for labeling learning data, which will set the stage for an ecosystem of higher-order applications of learning analytics
- Provide a standard way of measuring learning activities and effectiveness, which will enable designers and providers of curriculum to measure, compare and improve quality
- Leverage data science methods, standards and technologies
- Build upon existing 1EdTech standards
1.1 Scope
The project deliverable includes creating multiple artifacts to support the implementation and use of Caliper Analytics. These deliverables include:
- Creation of 1EdTech Metric Profiles to establish an extensible, common format for grouping learning activity data gathered across multiple learning environments.
- Creation of the 1EdTech Sensor API to define basic learning events and to standardize and simplify the gathering of learning metrics across learning environments.
- Creation of a reference 1EdTech Caliper Event Store primarily as a sample event persistence endpoint integration and to demonstrate the end-to-end event transmission and endpoint receipt / storage. Caliper does not formally include an open, standards based event store/LRS in its initial scope.
- Development and continuing support of 1EdTech Caliper Conformance Tools.
- Provide reference Caliper sensor implementations instrumented with Caliper sensors.
- Leverage and extend the 1EdTech Learning Tools Interoperability ® (LTI®) standard by enhancing and integrating granular, standardized learning measurement with tools interoperability and the underlying learning information models, inclusive of course, learner, outcomes and other critical associated context.
1.2 High Level Illustration
1.3 Nomenclature
Throughout this document, we use the following terminology when describing Caliper Analytics.
- Caliper: The name of the 1EdTech Caliper Analytics Learning Measurement Framework which provides a standard for representing, capturing, and marshalling metrics generated by learning activity and targeted for consumption by any conforming analytics store or service.
- Learning Activity: Any activity considered a component of a learning sequence presented in a digital learning environment (e.g. LTI tool) typically denoted as a lesson. Activities can be assignable and/or gradable as required.
- Caliper Sensor API: Defines and specifies the interaction between Learning Application and Caliper.
- Caliper Sensor: Code/library that assists in the measurement of Caliper Events using the Caliper Sensor API. Deployed into the Learning/Digital Application. Sensors are implemented in Java, Javascript, PHP, Python, Ruby and .NET.
- Metric Profile: The Information Model for Caliper – organized by Learning Activity Type.
- EventStore: System that stores event data emitted by Caliper Sensors.
1.4 References
1EdTech LTI®
1EdTech Learning Tools Interoperability® http://www.imsglobal.org/lti/
1EdTech LIS
1EdTech Learning Information Services http://www.imsglobal.org/lis/
1EdTech QTI®
1EdTech Question and Test Interoperability® http://www.imsglobal.org/apip/alliance.html
FOAF
FOAF http://xmlns.com/foaf/spec/
SCHEMA.ORG
Schema.org http://schema.org/docs/schemas.html
OPEN ANNOTATION
http://www.openannotation.org/
EPUB3
http://www.idpf.org/epub/vocab/structure/
JSON-LD
http://www.w3.org/TR/json-ld/
http://json-ld.org/
Activity Streams
http://activitystrea.ms/JSON-LD
xAPI
http://www.adlnet.gov/tla/experience-api/
2 Metric Profiles
The events and entities collected into metric profiles serve as the information model for Caliper. Metric Profiles are organized by learning activity type, for e.g., Assessment, Reading, Media, etc..
All of the events and entities defined in Caliper Metric Profiles are specified as Javascript Object Notation (JSON) and conform to the JSON for Linked Data (JSON-LD) format. As a result, events will specify both @context and @type element while entities will include an @id element in addition to specifying a @context and a @type. Note that these properties are not included in the tables below.
The sections below outline each Metric Profile specified for Caliper v1. As can be seen, for each Metric Profile, we define three types of information:
- Entities - These are the Entities that participate in Learning Interactions (e.g. Person, Assessment, Video, etc.)
- Actions - These are the actions performed as part of a Learning Interaction
- Event - captures the Entities and Actions performed as a “learning event”
2.1 Base Metric Profile
The base activity context is universally applicable to all activity types and contains common Entities and attributes used in the other profiles.
Entities
|
Property |
Data Type |
Description |
Source |
|
Entity (baseline for all other Entities) |
|||
|
name |
String |
Name of Entity |
schema.org/Thing |
|
description |
String |
Description of Entity |
schema.org/Thing |
|
extensions |
mapOf:extensions (String, String) |
Map of Extensions |
|
|
dateModified |
long |
time object was modified |
schema.org/CreativeWork |
|
dateCreated |
long |
time object was created |
schema.org/CreativeWork |
|
Property |
Data Type |
Description |
Source |
|
DigitalResource |
|||
|
objectType |
listOf: Assignable, Gradeable, Sharable … |
Type of Object Note: This property is not currently used and may be removed in a future version |
|
|
alignedLearningObjective |
listOf: associated learning objectives |
Associated Learning Objectives (Los) as context |
|
|
keywords |
listOf: keywords |
Associated Topics as context. List of Strings |
schema.org/CreativeWork
|
|
isPartOf |
DigitalResource Parent reference |
Parent/Referrer Learning Activity containing or linking to this Learning Activity |
schema.org/CreativeWork |
|
datePublished |
long |
time object was published |
schema.org/CreativeWork |
|
version |
String |
version or Entity |
schema.org/CreativeWork |
Event (baseline for all other Events)
|
Event Attribute |
Required |
Details (Type) |
Comments |
|
actor |
Yes |
User or System entity (FOAF:Agent) |
|
|
action |
Yes |
Defined per Metric Profile |
Action performed by Agent |
|
object |
Yes |
Entity (Variable based on profile)) |
|
|
generated |
No |
Entity (Variable based on Profile) |
|
|
target |
No |
Entity (Variable based on Profile) |
Optional, but recommended |
|
federatedSession |
No |
session id (String) |
Optional, but recommended |
|
edApp |
No |
edApp Entity |
Optional, but recommended |
|
group |
No |
Organization Entity |
Optional, but recommended |
|
membership |
No |
Membership Entitty |
|
|
eventTIme |
Yes |
Event time (ISO-8601) |
|
2.2Session Metric Profile
The Session Metric Profile is the vocabulary for tracking user session related activity/events.
Entities
|
Property |
Data Type |
Description |
Source |
|
Session |
|||
|
actor |
FOAF:Agent (user) |
Required. Agent for whom the Session exists |
http://xmlns.com/foaf/spec/ |
|
startedAtTime |
ISO-8601 timestamp |
Date that the session was started by Agent |
|
|
endedAtTime |
ISO-8601 timestamp |
Date that the session was ended |
|
|
duration |
xsd:duration |
Date that the assignable object is Active |
An xsd:duration (http://books.xmlschemata.org/relaxng/ch19-77073.html) |
Actions
|
Action |
URI |
Description |
|
Logged In |
http://purl.imsglobal.org/vocab/caliper/v1/action#LoggedIn |
Action indicating log in to start a session |
|
Logged Out |
http://purl.imsglobal.org/vocab/caliper/v1/action#LoggedOut |
Actor initiates end of session |
|
Timed Out |
http://purl.imsglobal.org/vocab/caliper/v1/action#TimedOut |
Session time out. |
Session Event
|
Event Attribute |
Required |
Details |
Comments |
|
actor |
Yes |
Person, EdApp |
|
|
action |
Yes |
From table above |
|
|
object |
Yes |
edApp (in most cases) |
|
|
generated |
Yes |
Session entity (from table above) |
|
|
target |
No |
Frame/webpage, etc. |
Optional, but recommended |
|
federatedSession |
No |
session id |
Optional, but recommended |
|
edApp |
No |
edApp |
Optional, but recommended |
|
group |
No |
Organization |
Optional, but recommended |
|
membership |
No |
Membership |
|
|
eventTime |
Yes |
Event time |
|
2.3 Reading Metric Profile
Vocabulary for activity/events related to the reading of textual content. The reading object is the main collection. Frame is the location within the content. Using the EPUB3 structural semantics, Volume is a component of a collection. Views are generated as part of or the entire object of the interaction with a Reading.
Entities
|
Property |
Data Type |
Description |
Source |
|
ePubVolume |
|||
|
ePubPart |
|||
|
ePartSubChapter |
|||
|
frame |
|||
|
index |
integer |
The numeric index of the location relative to sibling locations in the content |
|
Actions
|
Action |
URI |
Description |
|
Searched |
http://purl.imsglobal.org/vocab/caliper/v1/action#Searched |
Actor searches for a Digital Resource |
|
Viewed |
http://purl.imsglobal.org/vocab/caliper/v1/action#Viewed |
Actor views a Digital Resource |
|
Navigated To |
http://purl.imsglobal.org/vocab/caliper/v1/action#NavigatedTo |
Actor navigates to a Digital Resource |
Reading Event
|
Event Attribute |
Required |
Details |
Comments |
|
actor |
Yes |
Person, EdApp |
|
|
action |
Yes |
From table above |
|
|
object |
Yes |
Digital Resource in most cases |
|
|
generated |
Yes |
Search Result (for searched Action) |
|
|
target |
Yes |
Digital Resource f |
Optional, but recommended |
|
federatedSession |
No |
session id |
Optional, but recommended |
|
edApp |
No |
edApp |
Optional, but recommended |
|
group |
No |
Organization |
Optional, but recommended |
|
membership |
No |
Membership |
Optional, but recommended |
|
eventTime |
Yes |
Event started at time |
|
2.4 Annotation Metric Profile
The vocabulary for activity/events related to the annotation of a target resource. The annotation metric profile is a complimentary profile and can be combined with any learning activity such as a Reading Activity, an Assessment Activity, etc..
Entities
|
Property |
Data Type |
Description |
Source |
|
Annotation |
|||
|
annotated |
DigitalResource |
Required. The object being annotated |
|
|
BookmarkAnnotation |
|||
|
bookmarkNotes |
String |
The notes that the user entered with the bookmark |
|
|
HighlightAnnotation |
|||
|
textPositionSelector |
Object |
Selector which describes a range of text based on its start and end positions |
|
|
selectionText |
String |
The text that was selected |
|
|
SharedAnnotation |
|||
|
withAgents |
list of LISGroup or LISPersons |
The group or people that the annotation was shared with |
|
|
TagAnnotation |
|||
|
tags |
List of Strings |
List of Strings that the Actor tagged the content with |
|
Actions
|
Action |
URI |
Description |
|
Attached |
http://purl.imsglobal.org/vocab/caliper/v1/action#Attached |
Attach/detach a doc, media element, etc. with a target resource |
|
Bookmarked |
http://purl.imsglobal.org/vocab/caliper/v1/action#Bookmarked |
Record a position/place/location in a target resource |
|
Classified |
http://purl.imsglobal.org/vocab/caliper/v1/action#NavigatedTo |
Actor navigates to a piece of content |
|
Commented |
http://purl.imsglobal.org/vocab/caliper/v1/action#Commented |
A comment action associated/disassociated with a target resource |
|
Described |
http://purl.imsglobal.org/vocab/caliper/v1/action#Described |
A description or abstract is added or removed from an annotated resource |
|
Disliked |
http://purl.imsglobal.org/vocab/caliper/v1/action#Disliked |
Negative preference regarding the target resource |
|
Highlighted |
http://purl.imsglobal.org/vocab/caliper/v1/action#Highlighted |
Mark a selected segment of the target resource for particular emphasis |
|
Identified |
http://purl.imsglobal.org/vocab/caliper/v1/action#Identified |
Assign an identity to a target resource |
|
Liked |
http://purl.imsglobal.org/vocab/caliper/v1/action#Liked |
Indicate a positive preference regarding the target resource |
|
LInked |
http://purl.imsglobal.org/vocab/caliper/v1/action#Linked |
Link a resource related to the target resource |
|
Questioned |
http://purl.imsglobal.org/vocab/caliper/v1/action#Questioned |
Represents asking a question about the target resource(s). For example to ask for assistance with a particular section of text, or question its veracity |
|
Ranked |
http://purl.imsglobal.org/vocab/caliper/v1/action#Ranked |
Assignment of value or quality to the target resource(s). For example annotating an Annotation to moderate it up in a trust network or threaded discussion |
|
Recommended |
http://purl.imsglobal.org/vocab/caliper/v1/action#Recommended |
Recommend a modification to the target resource |
|
Shared |
http://purl.imsglobal.org/vocab/caliper/v1/action#Shared |
The annotated resource is shared to a target destination network and recipient |
|
Subscribed |
http://purl.imsglobal.org/vocab/caliper/v1/action#Subscribed |
The annotated resource is subscribed to (or unsubscribed from) a subscription service by a subscriber |
|
Tagged |
http://purl.imsglobal.org/vocab/caliper/v1/action#Tagged |
A tagging action is taken on a target reading element context |
Annotation Event
|
Event Attribute |
Required |
Details |
Comments |
|
actor |
Yes |
Person, EdApp |
|
|
action |
Yes |
From table above |
|
|
object |
Yes |
Digital Resource |
|
|
generated |
Yes |
Annotation |
|
|
target |
N/A |
|
|
|
federatedSession |
No |
session id |
Optional, but recommended |
|
edApp |
No |
edApp |
Optional, but recommended |
|
group |
No |
Organization |
Optional, but recommended |
|
membership |
No |
Membership |
Optional, but recommended |
|
eventTime |
Yes |
Event time |
|
2.5 Assignable Metric Profile
The assignable metric profile provides coverage for all activity types that are assignable to the learner for completion according to specific terms. Assignable is an interface that is implemented by the AssignableDigitalResource. Any Entity can implement AssignableDigitalResource and become “assignable”.
Entities
|
Property |
Data Type |
Description |
Source |
|
Assignable |
|||
|
dateCreated |
ISO-8601 timestamp |
Date that the assignable object was created |
schema.org/CreativeWork |
|
dateModified |
ISO-8601 timestamp |
Date that the assignable object was modified |
schema.org/CreativeWork |
|
datePublished |
ISO-8601 timestamp |
Date that the assignable object was published |
schema.org/CreativeWork |
|
dateToStartOn |
ISO-8601 timestamp |
Date that the assignable object is allowed to be started by Agent |
|
|
dateToActivate |
ISO-8601 timestamp |
Date that the assignable object is Active |
|
|
dateToShow |
ISO-8601 timestamp |
Date that the assignable object shows to students |
|
|
dateToSubmit |
ISO-8601 timestamp |
Date that the assignable object should be submitted |
|
|
maxAttempts |
int |
Maximum number of attempts |
|
|
maxSubmits |
int |
Maximum number of submits |
|
|
maxScore |
double |
Maximum score allowed |
|
|
Attempt |
|||
|
actor |
FOAF:Agent (user, system) |
Required. Agent that performed action. |
http://xmlns.com/foaf/spec/ |
|
assignable |
Assignable Object |
The object that the agent is attempting |
|
|
count |
int |
Number of attempts |
|
|
startedAt |
ISO-8601 timestamp |
Required time in milliseconds |
|
|
endedAt |
ISO-8601 timestamp |
Time in milliseconds |
|
|
duration |
xsd:duration |
|
|
Actions
|
Action |
URI |
Description |
|
Abandoned |
http://purl.imsglobal.org/vocab/caliper/v1/action#Abandoned |
Actor abandoned the assignment |
|
Activated |
http://purl.imsglobal.org/vocab/caliper/v1/action#Activated |
Assignable Digital Resource is available to Actors who are assigned it |
|
Competed |
http://purl.imsglobal.org/vocab/caliper/v1/action#Completed |
Actor completed the Assignable Digital Resource |
|
Deactivated |
http://purl.imsglobal.org/vocab/caliper/v1/action#Deactivated |
Assignable Digital Resource is no longer available to Actors who are assigned it |
|
Hid |
http://purl.imsglobal.org/vocab/caliper/v1/action#Hid |
Assignable Digital Resource is hid from the view of the Actors |
|
Reviewed |
http://purl.imsglobal.org/vocab/caliper/v1/action#Reviewed |
Assignable Digital Resource is reviewed |
|
Showed |
http://purl.imsglobal.org/vocab/caliper/v1/action#Showed |
The Assignable Digital Resource was shown to an Actor |
|
Started |
http://purl.imsglobal.org/vocab/caliper/v1/action#Started |
The Actor assigned started the assignment |
Assignable Event
|
Event Attribute |
Required |
Details |
Comments |
|
actor |
Yes |
Person, EdApp |
|
|
action |
Yes |
From table above |
|
|
object |
Yes |
Assignable Digital Resource,N/A |
|
|
generated |
Yes |
Attempt,N/A |
|
|
target |
N/A |
|
|
|
federatedSession |
No |
session id |
Optional, but recommended |
|
edApp |
No |
edApp |
Optional, but recommended |
|
group |
No |
Organization |
Optional, but recommended |
|
membership |
No |
Membership |
Optional, but recommended |
|
eventTIme |
Yes |
Event time |
|
2.6 Assessment Metric Profile
Provides coverage for Assessment, Quiz/Homework and similar assignable / gradable activity
Entities
|
Property |
Data Type |
Description |
Source |
|
Assessment |
|||
|
assessmentItems |
List of Assessment Items |
Required. |
|
Actions
|
Action |
URI |
Description |
|
Started |
http://purl.imsglobal.org/vocab/caliper/v1/action#Started |
The Actor started the Assessment |
|
Paused |
http://purl.imsglobal.org/vocab/caliper/v1/action#Paused |
The Actor paused the Assessment |
|
Restarted |
http://purl.imsglobal.org/vocab/caliper/v1/action#Restarted |
The Actor restarted the Assessment |
|
Submitted |
http://purl.imsglobal.org/vocab/caliper/v1/action#Deactivated |
The Actor submits the Assessment |
Assessment Event
|
Event Attribute |
Required |
Details |
Comments |
|
actor |
Yes |
Person, EdApp |
|
|
action |
Yes |
From table above |
|
|
object |
Yes |
Assignable Digital Resource,Attempt |
|
|
generated |
No |
Attempt, N/A |
Required in some cases |
|
target |
No |
N/A, Assignable |
Required in some cases |
|
federatedSession |
No |
session id |
Optional, but recommended |
|
edApp |
No |
edApp |
Optional, but recommended |
|
group |
No |
Organization |
Optional, but recommended |
|
membership |
No |
Membership |
Optional, but recommended |
|
eventTime |
Yes |
Event time |
|
2.7 Assessment Item Metric Profile
Covers the Actor’s interaction with assessment items.
Entities
|
Property |
Data Type |
Description |
Source |
|
Assessment |
|||
|
isTimeDependant |
boolean |
If the time to answer the question is important and should be recorded |
|
|
Response |
|||
|
assignable |
Object |
DigitalResource associated with the Response |
|
|
actor |
Object |
Agent associated with the response |
|
|
attempt |
Object |
Attempt Object associated with the Response |
|
|
startedAtTime |
ISO-8601 timestamp |
Required time in milliseconds |
|
|
endedAtTime |
ISO-8601 timestamp |
Time in milliseconds |
|
|
duration |
xsd:duration |
|
|
|
FilInTheBlankResponse |
|||
|
values |
ListOf |
List of values representing the response that was entered. |
ListOf |
|
MultipleChoiceResponse |
|||
|
Value |
String |
value choosen |
|
|
MultipleResponseResponse |
|||
|
Values |
ListOf |
values choosen |
|
|
SelectTextResponse |
|||
|
Values |
ListOf |
List of values representing the response |
|
|
TrueFalseResponse |
|||
|
value |
String |
value choosen |
|
Actions
|
Action |
URI |
Description |
|
Started |
http://purl.imsglobal.org/vocab/caliper/v1/action#Started |
The Actor started the Assessment Item |
|
Viewed |
http://purl.imsglobal.org/vocab/caliper/v1/action#Viewed |
The Actor viewed the Assessment Item |
|
Skipped |
http://purl.imsglobal.org/vocab/caliper/v1/action#Skipped |
The Actor skipped the Assessment Item |
|
Reviewed |
http://purl.imsglobal.org/vocab/caliper/v1/action#Reviewed |
The Actor reviewed the Assessment Item |
|
Completed |
http://purl.imsglobal.org/vocab/caliper/v1/action#Completed |
The Actor completed the Assessment Item |
Assessment Item Event
|
Event Attribute |
Required |
Details |
Comments |
|
actor |
Yes |
Person, EdApp |
|
|
action |
Yes |
From table above |
|
|
object |
Yes |
Assignable Digital Resource, Attemp |
|
|
generated |
No |
Assignable Digital Resource |
Required in some cases |
|
target |
No |
N/A, Assignable |
Required in some cases |
|
federatedSession |
No |
session id |
Optional, but recommended |
|
edApp |
No |
edApp |
Optional, but recommended |
|
group |
|
|
Optional, but recommended |
|
membership |
No |
Membership |
Optional, but recommended |
|
eventTime |
Yes |
Event time |
|
2.8 Outcome Metric Profile
Applies to all activity types that are assignable and also yield some measurable result and/or outcome
Entities
|
Property |
Data Type |
Description |
Source |
|
Result |
|||
|
assignable |
Object |
Assignable object associated with the Outcome |
|
|
actor |
Object |
Agent associated with the response |
|
|
normalScore |
double |
|
|
|
penaltyScore |
double |
|
|
|
totalScore |
double |
|
|
|
extraCreditScore |
double |
|
|
|
totalScore |
double |
|
|
|
curvedTotalScore |
double |
|
|
|
curveFactor |
String |
|
|
|
comment |
Object |
Assignable object associated with the Outcome |
|
|
scoredBy |
Agent that performed action. |
|
Required. Agent that performed action. |
Actions
|
Action |
URI |
Description |
|
Graded |
http://purl.imsglobal.org/vocab/caliper/v1/action#Started |
Actor or Software Application Grades the Assignment |
Outcome Event
|
Event Attribute |
Required |
Details |
Comments |
|
actor |
Yes |
Person, EdApp |
|
|
action |
Yes |
From table above |
|
|
object |
Yes |
Attempt |
|
|
generated |
Yes |
Result |
|
|
target |
Yes |
Assignable |
|
|
federatedSession |
No |
session id |
Optional, but recommended |
|
edApp |
No |
edApp |
Optional, but recommended |
|
group |
No |
Organization |
Optional, but recommended |
|
membership |
No |
Membership |
|
|
eventTime |
Yes |
Event time |
|
2.9 Media Metric Profile
This metric profile captures the interaction with audio/video systems.
Entities
|
Property |
Data Type |
Description |
Source |
|
MediaObject |
|||
|
Media Location |
|||
|
currentTime |
String |
The time value (from beginning of media) that indicates the current location |
Media fragment URI |
|
AudioObject |
|||
|
volumeMin |
String |
Minimum volume |
|
|
volumeMax |
String |
Maximum volume |
|
|
volumeLevel |
String |
Level of volume |
|
|
muted |
boolean |
Whether the Muted the audio |
|
|
VideoObject |
|||
|
ImageObject |
|||
Actions
|
Action |
URI |
Description |
|
Enabled Closed Captioning |
http://purl.imsglobal.org/vocab/caliper/v1/action#EnabledClosedCaptioning |
The Actor enabled Closed Captioning |
|
Disabled Closed Captioning |
http://purl.imsglobal.org/vocab/caliper/v1/action#DisabledClosedCaptioning |
The Actor disabled Closed Captioning |
|
Changed Volume |
http://purl.imsglobal.org/vocab/caliper/v1/action#ChangedVolume |
The Actor changed the volume |
|
Muted |
http://purl.imsglobal.org/vocab/caliper/v1/action#Muted |
The Actor muted the volume |
|
Unmuted |
http://purl.imsglobal.org/vocab/caliper/v1/action#Unmuted |
The Actor unmuted the volume |
|
Changed Speed |
http://purl.imsglobal.org/vocab/caliper/v1/action#ChangedSpeed |
The Actor changed speed |
|
Changed Volume |
http://purl.imsglobal.org/vocab/caliper/v1/action#ChangedVolume |
The Actor changed the volume |
|
Ended |
http://purl.imsglobal.org/vocab/caliper/v1/action#Ended |
The Actor ended the media interaction |
|
Jumped To |
http://purl.imsglobal.org/vocab/caliper/v1/action#JumpedTo |
The Actor jumped to a new media location |
|
Forwarded To |
http://purl.imsglobal.org/vocab/caliper/v1/action#ForwardedTo |
The Actor forwarded to a new media location |
|
Paused |
http://purl.imsglobal.org/vocab/caliper/v1/action#Paused |
The Actor paused the media |
|
Resumed |
http://purl.imsglobal.org/vocab/caliper/v1/action#Resumed |
The Actor resumed the media |
|
Rewound |
http://purl.imsglobal.org/vocab/caliper/v1/action#Rewound |
The Actor rewound the media |
|
Started |
http://purl.imsglobal.org/vocab/caliper/v1/action#Started |
The Actor started the media |
|
Changed Resolution |
http://purl.imsglobal.org/vocab/caliper/v1/action#ChangedResolution |
The Actor changed the resolution on the display |
|
Changed Size |
http://purl.imsglobal.org/vocab/caliper/v1/action#ChangedSize |
The Actor changed the viewer size |
|
Exited Fullscreen |
http://purl.imsglobal.org/vocab/caliper/v1/action#ExitedFullscreen |
The Actor exited full screen view |
|
Entered Fullscreen |
http://purl.imsglobal.org/vocab/caliper/v1/action#EnteredFullscreen |
The Actor entered full screen view |
|
Opened Popout |
http://purl.imsglobal.org/vocab/caliper/v1/action#OpenedPopout |
The Actor opened popout |
|
Closed Popout |
http://purl.imsglobal.org/vocab/caliper/v1/action#ClosedPopout |
The Actor closed popout |
|
Disable Closed Captioning |
http://purl.imsglobal.org/vocab/caliper/v1/action#DisabledClosedCaptioning |
The Actor disabled closed captioning |
|
Enabled Closed Captioning |
http://purl.imsglobal.org/vocab/caliper/v1/action#EnabledClosedCaptioning |
The Actor enabled closed captioning |
Media Event
|
Event Attribute |
Required |
Details |
Comments |
|
actor |
Yes |
Person, EdApp |
|
|
action |
Yes |
From table above |
|
|
object |
Yes |
Media Object |
|
|
generated |
N/A |
|
|
|
target |
N/A |
|
|
|
federatedSession |
No |
session id |
Optional, but recommended |
|
edApp |
No |
edApp |
Optional, but recommended |
|
group |
No |
Organization |
Optional, but recommended |
|
membership |
No |
Membership |
Optional, but recommended |
|
eventTime |
Yes |
Event time |
|
2.10 Illustration of Metric Profile Interactions
2.11 Engagement Scenario
As an example of just one common learning activity measurement, this scenario contains a list of Events and corresponding actions from the current set of Caliper Metric Profiles that indicate a baseline level of student engagement with Learning Activities. The determination of the what metrics constitute “engagement” may certainly differ across learning contexts so the metrics leveraged can be adjusted accordingly to align with any given engagement definition.
- In order to quantify engagement, three elements can be measured: (1) the overall duration of a “session” comprising multiple events; (2) the duration of engagement with each individual activity; and, (3) the total number of (each) type of event in a session.
- Each event has an “opening” action that indicates engagement. E.g. “started”. Other actions in the sequence (e.g. paused, resumed, completed) can also be included, including a “closing” action to indicate the end of the sequence. However, to simply get an indicator of (one or many) learner engagement with a specific Learning Activity, the opening action could be sufficient.
- All attributes/objects (e.g. actor, action, object, startedAtTime, endedAtTime, duration, etc) of Events are implicitly part of the Engagement Scenario.
|
Event |
Action |
Metric Profile |
Comments |
|
SessionEvent |
loggedIn |
Session |
|
|
Navigation Event |
navigatedTo |
Mulitiple |
Common across all Learning Activity types |
|
ViewedEvent |
viewed |
Reading |
|
|
AnnotationEvent |
All |
Annotation |
|
|
MediaEvent |
started, [paused,resumed,etc] |
Media |
|
|
AssignableEvent |
started [completed,submitted] |
Assignable |
|
|
AssessementEvent |
started, [submitted] |
Assessment |
|
3 Sensor Implementation Details
To insure that all the of the Sensor’s are producing the same JSON-LD and for ease in implementing the Conformance Testing Framework, the Caliper Common Fixtures project was created. Version 1.0 aligned Caliper Common Fixtures are included in the Appendix of this document.
3.1 Java Sensor
The Reference Implementation of the Java Sensor is located in the chart below:
Directions to install and build the project are located in README of the project.
3.2 Javascript Sensor
The Reference Implementation of the Javascript Sensor is located in the chart below:
Directions to install and build the project are located in README of the project.
3.3 PHP Sensor
The Reference Implementation of the PHP Sensor is located in the chart below:
Directions to install and build the project are located in README of the project.
3.4 Python Sensor
The Reference Implementation of the Python Sensor is located in the chart below:
Directions to install and build the project are located in README of the project.
3.5 Ruby Sensor
The Reference Implementation of the Ruby Sensor is located in the chart below:
Directions to install and build the project are located in README of the project.
3.6 .NET Sensor
The Reference Implementation of the .NET Sensor is located in the chart below:
Directions to install and build the project are located in README of the project.
|
3.1 |
|
|
3.2 |
|
|
3.3 |
|
|
3.4 |
|
|
3.5 |
|
|
3.6 |
4 Caliper Transport Support
In order to support some additional requirements related to the work being done with Caliper in the 1EdTech HED Analytics group (formerly RAM) as it relates to requirements to leverage a more minimal event with a free-form payload, the current released Caliper v1.0 sensorAPI and CaliperEvent implementation and conformance validation support will be revised to a “transport” level of use. This essentially means that only the most minimal and generally applicable event structure, nomenclature and supporting properties will be leveraged and the more comprehensive CaliperEvent via metric profile modeled event definition and context will not be required.
4.1 Transport-Only Scope for Sensor API and CaliperEvent
Events must include:
- The CaliperEvent structured “triple” of Actor, Action and Object (must adhere to Caliper naming conventions/vocabulary)
- Actor: object with the @id attribute required.
- Object: object with the @id attribute required.
- Action: a string with the @type attribute required (recommended format URI)
- eventTime: the event ISO timestamp
- Optional
- generated object (optional, If specified, must have an @id attribute. e.g. used for attempt, result, response etc)
- target object (optional, If specified, must have an @id attribute)
4.2 Caliper-Transport Event Sample Fixture
{
"sensor": "https://example.edu/sensor/001",
"sendTime": "2015-09-15T11:05:01.000Z",
"data": [
{
"@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context",
"@type": "http://purl.imsglobal.org/caliper/v1/Event",
"actor": {
"@id": "https://example.edu/user/554433",
"@type": "http://purl.imsglobal.org/caliper/v1/lis/Person" },
"action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Viewed",
"eventTime": "2015-09-15T10:15:00.000Z",
"object": {
"@id": "https://example.com/viewer/book/34843#epubcfi(/4/3)",
"@type": "http://www.idpf.org/epub/vocab/structure/#volume" }
}
]
}
4.3 Caliper Transport and Conformance/Compliance Validation
With this more recent transport-only support, the following levels of conformance test compliance and/or certification will be enabled and supported:
- Caliper-Transport Compliance: compliance-only validation of the SensorAPI and minimal CaliperEvent structure defined by the transport-only requirements….therefore, no 1EdTech certification will be issued for this level of compliance with Caliper.
- Caliper-Event Conformance: compliance and certification for implementations of the SensorAPI and one or more declared supported metric profiles. (Note: this is the primary and default Caliper implementation leveraging the SensorAPI and the CaliperEvent modeled event data and contextual payloads)
- Caliper-LTI Conformance: compliance and certification for implementations of LTI Provider and/or Consumer which include support for the defined Caliper Service Profile service extension. (Note: planned for v1.1 release to with completion of the related v1.1 ref implementation)
5 Use Cases for Caliper Analytics
This section describes common scenarios for the Caliper Specification.
5.1 Caliper Enabled Engagement Sequence
This profile contains a list of Events and corresponding actions from the current set of Caliper Metric Profiles that indicate student engagement with Learning Activities.
|
Use Case Title: |
Example Engagement Sequence |
|
Brief description: |
A student’s engagement with a common learning sequence |
|
Level: |
Summary |
|
Actors: |
Student LMS Electronic Reading Educational App Quizzing Application |
|
Basic Flow of Events: |
|
5.2 Example Reading Sequence
|
Use Case Title: |
Example Reading Sequence |
|
Brief description: |
A student in a course interacts with two readings from two reading eduApps. In the process of interacting with the apps, she performs various reading and annotation activities. These are defined in the Caliper Reading and Annotation profiles respectively. |
|
Level: |
Summary |
|
Actors: |
Student LMS Electronic Reading Educational App 1 Electronic Reading Educational App 2 |
|
Basic Flow of Events: |
|
5.3 Example Assessment Sequence
|
Use Case Title: |
Example Assessment Sequence |
|
Brief description: |
Student in a course interacts with assignable entities within the course. The assignable here is an Assessment. In the process of interacting, she performs various assignable and assessment related interactions. These are defined in the Caliper Assignable, Assessment and Outcomes profiles respectively |
|
Level: |
Summary |
|
Actors: |
Student LMS Assessment EduApp Grading Engine
|
|
Basic Flow of Events: |
|
5.4 Example Media Sequence
|
Use Case Title: |
Example Media Sequence |
|
Brief description: |
A student in a course interacts with multi-media entities within the course. The events and entities referenced here are defined in the Caliper Media Metric Profile |
|
Level: |
Summary |
|
Actors: |
Student LMS Media Player |
|
Basic Flow of Events: |
|
6 Conformance Testing Details
6.1 Caliper Conformance Testing Framework Diagram
6.2 Caliper Conformance Testing Details
Caliper Conformance Tests is a set of tooling to help verify if events are aligned with the Caliper format and to show if the metric profiles are complete.
For more details and instructions for Conformance Testing, please refer to the 1EdTech Caliper Conformance and Certification Guide available on the Caliper Developer Site
7 Appendix
7.1 Caliper Common Fixtures JSON
The Common Fixtures resource can be found and referenced as part of the Caliper codebase (https://github.com/IMSGlobal/caliper-common-fixtures-public).
About This Document
Title: 1EdTech
Editor: Mary Millar, Elsevier
Co-chairs: Chris Vento, Intellify Learning, Chris Millet, Penn State University
Version: Final v1.0
Version Date: 10 October 2015
Revision Information: Version 1.0 Final
Document Location: http://www.imsglobal.org/caliper/
List of Contributors
The following individuals contributed to the development of this document:
|
Viktor Haag |
D2L |
|
Mary Millar |
Elsevier |
|
Prashant Nayak |
Intellify Learning |
|
Chris Vento |
Intellify Learning |
|
Anthony Whyte |
University of Michigan |
Revision History
|
Version No. |
Release Date |
Comments |
|
Final v1.0 |
15 October 2015 |
Final formal release of this specifcation. |
|
|
|
1EdTech Consortium, Inc. (“1EdTech”) is publishing the information contained in this document (“Specification”) for purposes of scientific, experimental, and scholarly collaboration only.
1EdTech makes no warranty or representation regarding the accuracy or completeness of the Specification.
This material is provided on an “As Is” and “As Available” basis.
The Specification is at all times subject to change and revision without notice.
It is your sole responsibility to evaluate the usefulness, accuracy, and completeness of the Specification as it relates to you.
1EdTech would appreciate receiving your comments and suggestions.
Please contact 1EdTech through our website at http://www.imsglobal.org.
Please refer to Document Name: 1EdTech Caliper Analytics Implementation Guide v1.0
Date: 15 October 2015
