Sharebar?

Learning Impact Blog


Community leadership for more effective use of technology in service to education

With IMS’s recent announcement of the upcoming e-assessment interoperability challenge we thought it would be a good time to discuss electronic assessment. Here is a Q&A with Rob Abel of IMS Global. Feel free to post additional questions and Rob will answer them (if he can)!

Q1: Is it time for electronic assessment in education?

A1: Yes, paper tests are more difficult to administer, take longer to process, are more prone to error and are not able to provide timely data to help improve instruction. Compared to a situation where paper textbooks may still have some usability advantages over digital e-books, paper assessments have no advantage at all over e-assessment.

Q2: Can e-assessment be used for summative or formative testing?

A2: Both.  E-assessment can be used for pure “high stakes test taking” scenarios as well as intermingled throughout other learning activities for formative assessment.

Q3: Is interoperability of assessment items important?

A3: Yes - very. In general digital assessment enables new forms of collaboration. For instance, in various countries around the world there is a desire to enable school organizations to collaborate on item development – since many schools are testing on the same subjects. Standard formats for assessment items enables collaboration on/exchange of items without every organization needing to use the same software platform for item creation and/or delivery. It is becoming pretty clear with historic collaborations such as the U.S. states on the Race to the Top Assessment initiative that the era of the “single delivery platform that outputs pdf” is coming to an end. With interoperability of assessment items enabled by standards there is no reason to be locked into a single vendor solution. Across the assessment community replication of effort goes down, investment in proprietary solutions ends and more investment is focused on innovation.

Q4: Does IMS have standards and a community focused on assessment interoperability?

A4: Yes.  IMS has two related standards that the assessment community worldwide should be making use of. The first is QTI (Question and Test Interoperability) and the second is APIP (Accessible Portable Item Protocol). QTI enables interoperability of assessment items and tests. The latest version is v2.1 which is the one that the assessment community is rallying around. A subset (profile) of an older version of QTI, v1.2, is used in Common Cartridge, which is a format for importing and exporting content into/out of learning platforms. APIP adds accessibility constructs to QTI to enable electronic delivery of a variety of accessible assessments.

Q5: What about other types of interoperability that might enable more effective use of e-assessment?

A5: Yes. There is a very compelling need to use interoperability standards to enable assessment software platforms to “plug into” or connect with other software systems. So, this is the “assessment software product” as an LTI (Learning Tools Interoperability) tool provider, enabling the assessment platform to be seamlessly “launched” from a host system (like a learning management system). This type of “plugging in” can be useful in both formative and summative scenarios (depending on how the later is administered). We see at least four types of assessment products beyond the state level large-scale assessment that will benefit from this type of interoperability:

  • Standard quizzing/test authoring and delivery software that are typically used already with learning platforms
  • The increasingly popular “homework applications” or “adaptive tutoring applications” can be also be viewed as formative assessment platforms.
  • Classroom test creation and scoring systems – yes, including those using paper and pencil
  • Assessment tools used for competency-based degree programs, such as those used by Western Governors University.

Q6: What about interoperability of assessment data?

A6: This of course is also very important. QTI describes formats for item data – which describes how test takers answer questions. The latest IMS work on analytics – the IMS Caliper Learning Analytics Framework (see blog Q&A) - will leverage the QTI data formats as well as other assessment-related formats (e.g. gradebook data). Thus, assessment data can be provided “back” to a learning platform, an assessment delivery platform or to an analytics store.

Q7: What about authentic assessment in the classroom or project-based learning?

A7: Any type of educational assessment, including e-assessment, is just a tool. It is one source of input. In our opinion assessment should be used to improve teaching and to improve learning. Thus, e-assessment plays an important role because it can provide real-time or near real-time feedback in a very transparent way – on a question by question basis (QTI enables such feedback), for computer adaptive testing or simply faster processing of an entire quiz or test. And that feedback can go to teachers, students, parents, etc – whatever makes the most sense. And, initiatives like Race to the Top Assessment are folding teacher evaluation of various “performance events” into the assessment mix. Mobile platforms and interoperable apps could obviously have an very important and innovative role to play in that regard as well as all types of assessment wrapped into apps or otherwise. We’ve already seen some fascinating use of QTI in the mobile setting via the Learning Impact Awards.

Q8: Why has IMS announced a Worldwide Assessment Interoperability Challenge?

A8: Use of interoperability standards such as QTI in the past has been rather flakey in that each supplier implemented different versions and different subsets of functionality. Very few assessment product providers provided feedback to IMS to enable the issues to be resolved.  As a result, interoperability was limited.  Things have turned around radically in the last few years in that IMS now has some 25 or so world-leading providers of assessment products actively involved in implementing QTI and/or APIP. As a result, IMS has been able to finalize these specifications and conformance certification tests that will result in high levels of interoperability. The “challenge” is our way of saying to the world that we have a very strong core set of suppliers who have agreed to achieve conformance certification together over the next few months. Please come and join in for the good of your product development efforts and the good of your customers who desire interoperability that really works.  The extra added “bonus” for participating is entry into the annual IMS Learning Impact Awards under special assessment product categories. Details on the “challenge” are here: http://apip.imsglobal.org/challenge.html

Q9: What if a region of the world wants to work with IMS on a regional profile of QTI or APIP?

A9: Yes, IMS is set up to facilitate that and is in fact in partnership in the Netherlands for the last two years on such an effort regarding national exams.  Feel free to send us an email at assessmentchallenge@imsglobal.org

Q10: What do you see for the future of e-assessment?

A10: We are at the very beginning of a long road ahead filled with many exciting product opportunities.  As with many of the other IMS standards, like Common Cartridge and LTI, we are going to see a very dynamic evolution based on market needs of QTI and APIP. For instance, one of the other application areas we are working on at the moment is QTI application to e-textbooks. E-assessment will permeate every aspect of digital learning materials and activities – with an emphasis on adaptive testing to help pinpoint where additional alternative materials and activities are needed. And, with the undeniable trend toward competency-based learning paths and credentialing the need for better assessment is increasing. As with all of the IMS focus areas the key will be for the technology of assessment to “get out of the way” and be simple and easy to use and benefit from.  

This blog post is an interview with IMS’s Rob Abel to get to the bottom of IMS’s recent announcement of its Caliper Analytics Framework.

See related post on "small data".

Feel free to post additional questions and Rob will answer them (we hope)!

Q1: Is this project/announcement a big deal?

A1: We’ve got a lot of very impactful stuff going on in IMS these days, but enabling widespread adoption of analytics is one of the top priorities of IMS – with a mandate coming right from the IMS Board of Directors. But, perhaps more importantly, if we want to gain the full potential benefit of analytics and dashboards in education we need to make sure it is relatively easy to enable the transmission of data from any applications that can provide useful data. Interoperability standards, if done correctly, can help enable this.

Q2: There is lots of work going on in analytics, dashboards, etc.  Why is this a credible entry into the analytics space by IMS?

A2: Several reasons. IMS has had a relatively unique focus on one of the potentially more fruitful but challenging data collection areas of learning applications and platforms. IMS knows this turf well and brings a large critical mass of members that cover a wide range of product categories and institutional needs. IMS also has a large installed base of applications already using its core standards, such as LTI (Learning Tools Interoperability), Common Cartridge, LIS (Learning Information Services)  and QTI (Question and Test Interoperability). In other words, data is already flowing via these specifications, which are providing conduits upon which more data can ride.

Q3: What is the focus of the analytics initiative versus other IMS specification work?

A3: IMS has a bunch of fast moving task forces and leadership groups that work on applications of standards. This analytics work is coming from such a group that has been in existence about a year, but leveraging off of years of work by IMS in specifications for outcomes data for a variety of purposes – ranging from scores to gradebooks to assessment item data.  The purpose of the analytics effort is to actualize many implementations of analytics feeds in as many products as rapidly as possible, adding a few new bits and pieces, but largely leveraging existing work. In fact, the first proof of concept demonstrations will come very soon – at the next IMS quarterly meeting the week of November 4.  There will be a Summit day there, Thursday, November 7, that will also focus on analytics – both the institutional and supplier perspectives.

Q4: What is the relationship with IMS LTI (Learning Tools Interoperability)?

A4: There’s a short answer and a longer answer.  The short answer is that we expect this analytics work to finalize some work on outcomes (namely a very robust gradebook) that has been proposed for LTI & LIS for awhile but not officially released yet and that we expect the proliferation of LTI-enabled apps and learning platforms to be a natural starting point for data exchange. The longer term has two components. The first is that we are leveraging the work of IMS members in specific LTI app/product categories to help develop the Learning Metric Profiles referenced in the Caliper whitepaper. The second is that we will be introducing a variety of LTI Services that will enable data to go to many different destinations from many sources whether or not LTI was used as the launch mechanism for an app. So, for instance, enabling apps to send data to an analytics storage, dashboard or personal data vault whether or not it was launched by an LMS/learning platform.

Q5: Why is this project developing and releasing the Sensor API?

A5: APIs can make implementation a little easier – especially if a large number of suppliers use the same APIs. IMS is now releasing APIs as best practices with many of its specifications now. Please note that an “API” is by definition programming language specific and a good standard is not.  The standard is the underlying guts – that’s the hard part.

Q6: What if a product company has already developed an API for some category of data transmission – can that still be used with Caliper?

A6: Maybe. One of the cool things about the IMS specs and the development process behind them is that we can work with leading suppliers who already have services/APIs to see if we can “map them” on top of the IMS specifications. You may have developed some APIs that are now experiencing good market adoption for a specific type of service. IMS can potentially work with your organization to harmonize that with Caliper and the LTI services. Please contact us at: CaliperFramework@imsglobal.org

Q7: What if IMS can’t get suppliers to agree on the Learning Metric Profiles?

A7: Well, we wouldn’t be doing this if we were not already seeing some excellent convergence. But, we also want to encourage and fully expect there to be extensions, both public and private, that IMS will capture in a registry.  Thus we can have the stuff that everyone agrees on and the stuff that is new, above and beyond that is either publically sharable or not. That’s how innovation in data and analytics is enabled by all of this.

Q8: What about applications that are kind of out of the learning domain, like CRM (Customer Relationship Management) systems?

A8: We see absolutely no reason why Caliper cannot add Metric Profiles for classes of systems like this and add into the mix. The Caliper Framework should be applicable to almost any type of system.

Q9: What if my analytics product wants to suck in every possible data and user interaction possible?

A9: Yes, big data. If you want to do that across more than one system you still need an agreed upon analytics feed. Caliper will cover that, even if a private solution is needed at first (see A7 above).

Q10: Will U.S. K-12 initiatives such as InBloom or Ed-Fi benefit from this work?

A10:  Yes, they certainly could! Caliper data can go to/from anywhere.  

At the 2013 Learning Impact conference I presented a keynote “Innovation, Disruption, Revolution – Oh My!”  I chose this topic because the degree of hype about “disruption” in education seems to be at an all time high right now. BTW it's amazing how well the Gartner Hype Cycle fits the Wizard of Oz! From Rob Abel's Learning Impact Keynote: Innovation, Disruption, Revolution Oh My!

From Rob Abel's Learning Impact Keynote: Innovation, Disruption, Revolution Oh My!

Excitement about the role of technology in improving education is a good thing as far as I’m concerned. Education is a segment that needs disruptive innovation. To me the hype around things like MOOCs represents a longing by many for “a better educational future” – presumably involving lower costs to students and better career/life fulfillment, not to mention better global citizens needed to solve our global challenges. Let’s face it - there is a general sense that the current education system is not up to the challenges of the future. And, it’s not clear how we get from “here” to “there.”

But, as leaders in the education segment we do need to get better at understanding where we have been and where we are going, what constitutes innovation and/or disruption that is worthy of investment?  Are you an investor? I would argue that any individual that is putting time into educational technology leadership at any level is an investor, but certainly institutions that are spending on innovation, and yes, venture capital investors (investors in this segment, including some pretty big names, are making some bad decisions right now about where they are putting their money – but this post is more about how institutions should decide to invest their resources).

In IMS we try to ferret out “winners” by looking at criteria for something we call “Learning Impact” – which is defined by a set of judging criteria we use in our annual Learning Impact Awards (LIAs). You can think of it as evidence that the application of technology in an educational setting has had a clear impact on access, affordability and/or quality. We’re pretty proud of this program because there is simply no way to win with hype. But, in general the whole annual Learning Impact event is about understanding where the innovation really is.

Right now there are quite a few over-hyped activities in the education segment.  I would include in these MOOCs, Common Core State Standards, analytics, badges and open educational resources (OER). Sorry if I tipped over one of your sacred cows there!  Over-hyped does not mean that something good will not come from these developments. It just means that they are being portrayed as more significant in terms of ability to “disrupt” education than they will be capable of delivering on. As with most hyped innovations, eventually some aspects of the innovation “survive,” crossing the chasm to productive advancement of the industry. The challenge for innovators and investors is to utilize critical thinking to rationalize what will sustain and what the real Impact will be.

MOOCs are perhaps everyone’s favorite example of hype right now. It’s difficult to imagine how something could have achieved more hype in the higher education segment – and they are very clearly striking a nerve for being potentially highly disruptive. Literally every day MOOCs are in the headlines – and smart marketers are trying to jump on the MOOC bandwagon even as that bandwagon morphs continually to address the glaring deficits of the MOOC model and quite frankly, more failures than successes at this point.

Clayton Christensen, the leading expert on “disruptive innovation,” has written at least two books specifically focused on education. According to Christensen’s disruptive innovation theory markets are disrupted when new entrants figure out an innovative way to provide a “simpler” product to a wider set of buyers at a more affordable price.  Since the simpler product is actually what the broader market prefers (simpler means more usable, more effective for the desired purposes) the product is highly disruptive – better product at lower price point.  And thus, these new entrants change the market behavior substantially.  Jim Farmer does a very thorough job of digging into the theories as applied to MOOCs here.

While I, like many other silicon valley entrepreneurs, have found Christensen’s original formulations on disruptive innovation descriptive of what is generally seen in the high tech world, and a useful thought framework, well, the problem is that these formulations have not been at all predictive for education – and that is the acid test for using theory for strategic gain – is it predictive?  Writing about what happened in the past and putting a framework around it is great – but if it doesn't help predict the future it doesn’t especially help entrepreneurs or us “investors.”

The innovators in the education segment have NOT disrupted the status quo significantly so far. While Christensen predicted in 2008’s Disrupting Class great disruptions in the segment from online learning, the reality so far is that greater adoption of online learning has continued as expected but not very disruptive: price points for education continue to rise ahead of inflation and while online education continues to grow it is largely seen as reflecting traditional models rather than a disruption.  And, while online/blended models have certainly improved access – well, the percentage of the population that has achieved credentials has been very level.  I wrote a paper about what the technology impact in higher education back in 2005-7 - and the situation is not significantly different today.

Christensen was recently quoted in an interview as stating that higher education institutions are going to be in real trouble 5 years from now. However, he has not made it clear why things will be different in the next 5 years versus the last 5 years since Disrupting Class was published. I do agree that “buyers” (students/parents) are in fact getting smarter about looking for lower cost options as well as attempting to understand the value of a higher education degree.  But I would argue that we are nowhere near a true disruption (rise) in the number of participants in higher education.

While it is very valuable to have any great thinker on business strategy analyzing education and providing insights from other industries that might apply (like Christensen), education leaders need to do their own critical thinking about these formulations of “disruptive innovation” in the education segment.

I’d like to provide three key factors that IMHO are needed to be understood if to  understand disruption in education. While these factors may be relevant to understanding MOOCs, they can be applied to other hype areas as well. Hold on to your hat here as these are things that we don’t hear much talk about in the education space – especially when you go to meetings on one of the hyped topics or even to investor conferences!

  1. Education is a complex services business in which quality is difficult to define. Disruption in the education space requires better service models that are built around improved educational program quality. Comparison of education to the disruption of the steel industry by mini-mills (a connection some have made because Christensen uses this as a classic example of disruption) is not a valid comparison. Disruption in education is not about replacing the low end of a well-defined product. It’s about redefining quality in a much more complex world of knowledge than that from which most current educational models were designed.  So, for instance, a true disruption in education would be highly desirable/effective K-masters degree in 15-16 years versus the current 19 years (ideally that meets the needs of under performing populations as well as traditionally successful populations).
  2. The next phase of true progress will be to come out of the current era of massification into a new era of more real world relevant and personalized educational pathways.  Massification of educational experiences will not be the ticket to success in the next wave of educational models. It seems that many of the entrepreneurs and investors in the education space fail to realize that we have already been in the era of massification of education for the last 30 years or so in developed countries. The disruption in terms of content will need to be content that enables educational experiences that are up-to-date (read “relevant”), adaptive to the interests of the learner, easily adaptable by teachers and yet thorough in terms of teaching the educational foundation that are defined via #1.  No offense to our friends at Coursera (one of the growing number of MOOC providers), but “course era” is an especially off the mark name for describing the future IMHO.  It is NOT a course era now in education nor will it be in the future. It is the same era that it has been: which is one in which a distinctive program of high quality education will be highly valued. It's just that we need to do a better job getting such distinctive programs to the currently underserved populations at a better price point. Yes, there will be different sources of supporting digital resources that will help enable the redesign of educational delivery (potentially a role for MOOC courses). As discussed in other LI blog posts (here & here) digital resources will need to be in a form of a highly usable toolkit for faculty and learners.  But, these supporting resources have a VERY long way to go to meet the needs of learners. And, the more available (i.e. free, open, massive, etc) a course or content becomes the faster it will become commoditized.
  3. Education is the ultimate “long tail” market with a growing proliferation of high value niche providers and boutique programs. This is only going to increase in the future. Contrary to other recent online phenomena like facebook or twitter, education is not a “winner take all” market. I was at an education investor conference earlier this year where a leading investment advisor made the statement that education is the “ultimate winner take all market.” Education, if done correctly, is life success enabling. The more unique and distinctive your educational experience is, the more valuable it is. The ability to produce knowledge is the key currency in the current and future global economy. There is no distinctiveness to attending “Massified University.” And, a credential from a massive provider will most likely be such a commodity that most will prefer not to waste their time on it (other than for pure fun or reference). We need many more niche-oriented institutions that provide specialized, career-enabling and life-enabling education. Note that even large public institutions, while having many students, can engage this philosophy to create a large number of differentiated programs of study.   Therefore, the “disruptive technology platforms” for education will need to enable great diversity. Diversity in terms of delivery models and blending of high touch with personalized self-service.  Disruptive platforms will also need to enable seamless integration among cooperating providers of the various components of a solution – meaning close partnership among institutions as well as innovative learning tools.  Old style Web 2.0 thinking of the single pervasive platform or the single way of analyzing data will not work for the future of education. Education is not that simple (sorry!).

Are MOOCs potentially promising innovations? Yes. They are clearly an evolution of the trend toward pervasive online/blended/more flexible educational models/flipping the classroom. Will they disrupt education?  Not on the current trajectory. They pretty much fly in the face of the three tenants above.

But, there are some potential ways in which MOOCs could be disruptive in a more limited way.

I think MOOCs have the potential to be disruptive on the low end of the education market as a new model for delivering “open university style education.”  For example, today’s MOOCs could be an appealing alternative to the content portion of massive open universities around the world, featuring star professors from highly ranked universities. And, such populations of learners could probably support an advertising/low course fee model of consumption. So, open university providers get a “pay per use” version of content (versus the larger investments they must make today for tuition) that is likely better than what they are offering today, which if subsidized by advertising (net revenue equals small pay per use from open university provider coupled with advertising revenues) could equate to substantial revenues over a large numbers of users with similar interests (as indicated by the topic of study).

In such a model MOOCs could also double as a new type of “tutorial” reference materials. Very much like Kahn Academy, which is often referred to as a MOOC now even though it existed for a few years prior to the term.

Would this application of MOOCs be disruptive? Well, if you consider displacement of open universities and/or a new business model for them disruptive, then yes.  This approach could potentially disrupt the open university market worldwide and create a much larger interest in open university derivative models as a way to learn more about a particular topic and/or as preparation for entry to other universities.  Will they substitute for those universities – no. Such a disruption will not radically change the efficiency of higher education segment at all.

Is there a strong motivation for current providers of education to engage in this model? Unclear. As previously discussed the more you commoditize an educational experience the less valuable the education is. As discussed above the next stage in education is greater diversification, not massification. Personally I would like to see existing institutions respond to the three factors above by rolling out new “institutes” with new types of degree programs that meet the evolving needs of society.

Should open universities or other new entrants that wish to compete in that space try a MOOC/advertising/low course fee supported model?  I hope so.  That would be a good fit with the “course-focused” strength of MOOCs.  It would also provide a potential revenue producing and marketing outlet for institutions and their “star teachers.” Of course there are the normal challenges of achieving accreditation from the countries/states as needed to prove value to the potential students.

So, hooray for the MOOC phenomena in terms of making claims about needed innovations such as scale, analytics and the potential power of star teachers that will help accelerate the innovation trend toward online/blended that we are already on! And, congratulations to the leading institutions willing to make a space to try out the MOOC innovations as another set of tools that might be leveraged in the quest for improved education.

However, I would say that if you don’t know at this point why you are investing (in terms of what impact you expect will sustain at your institution, and therefore what the return on your investment is likely to be) then you are not applying the level of critical thinking you need to. Personally I would be asking MOOC providers to take the risk in terms of proving the validity of the market/business model (such as the ideas around open university displacement) – and not taking on that risk for my university. 

[Note: In a future blog post I will take a whack at what I think will be the sustainable innovations that might cross the chasm coming out of some of the hype items mentioned above: MOOCs, Common Core State Standards, analytics, badges and open educational resources (OER).]

See more of Rob Abel’s views on educational innovation throughout the Learning Impact blog as well as this feature interview with Anthony Salcito of Microsoft.  

Special Blog from AACC Annual Convention @Comm_College ‪#AACCAnnual

Regardless of what your view on how “liberal” a program of study at a college should be, it seems to be a fair assumption that colleges should help qualify students for a good job and great career. Especially considering the high debt loads that students in the U.S. are incurring to get a college degree – they need a good job to pay for it.

What is the role of interoperability in educational data? As I have posted elsewhere, IMS is working diligently on interoperability of both big data and small data. We are aligning ourselves closely with the needs of institutions leading the charge on competency-based education credentialing. And, we are strong supporters of the U.S. Department of Education and White House “My Data Button” initiative.

Today I had the privilege of moderating a panel on “closing the gap” between college offerings and the world of work. Our distinguished panelists were:

Debra Derr, President, North Iowa Area Community College

Richard Carpenter, Chancellor of Lone Star College System, and Chair of the Texas Association of Community Colleges

Shah Ardalan, President, Lone Star Community College, University Park

North Iowa Area Community College and Lone Star College System (Houston) represented a great range in scale in this conversation, with NIACC being a small community college and LSCS being one of the largest and fastest growing in the U.S.

However, despite the range in size, the best practices were in agreement:

  1. Understand what your student and faculty expectations are with respect to use of technology and technology innovation.
  2. Partner with organizations who have knowledge and expertise to avoid having to reinvent the wheel in terms of deploying new technology.
  3. Build close ties to local industry to understand the needs of employers.
  4. Provide better resources to help students understand employment opportunities, and in general what in the world their degree is qualifying them for.
  5. Move more toward competency-based programs and student documents (evidence of competencies) that can be owned by the student to be used in their quest to match career opportunities.

Richard Carpenter challenged the audience of community college leaders to transform what colleges can do for students by enabling students to “own the student record.” This is a massive paradigm shift from the last 40 years of institutions being the owner of the student data.  But today’s panel questioned whether this is good enough for the future.

Obviously there is nothing wrong with institutions being the keepers of authoritative records about student achievement. The problem occurs when students and parents realize that they have paid for an education for which they have little to show except a transcript. Thus, the challenge by Chancellor Carpenter, and echoed by the other panel participants, is that institutions need to help students understand opportunities, create and organize the artifacts from their learning according to critical competencies, and ultimately enable students to “take this with them” throughout their lives.

Lone Star College has championed a new service called the Education and Career Positioning System – which has been launched as an online service for students in which they can own their data.

IMS Global has been working hand-in-hand with Lone Star on this initiative because we believe this is an absolutely critical element of the IMS Open Digital Innovation Revolution in Education, namely opening up the campus systems so that students can connect their academic accomplishments to career and academic opportunities. Obviously IMS open standards can play an important role in opening up the data and artifacts created in a myriad of educational software for export to the student record: the one owned by the student of the future.

Tags:

A special featured keynote by Bob Mendenhall, President of Western Governor’s University at Learning Impact 2013

It really doesn’t matter who you talk to in the education field. Literally all agree that doing a better job of understanding competencies is the way that education needs to move. It’s about what you know and what you can do, rather than what course you took and what grade you got in that course.

Western Governors University (WGU) is the recognized leader in competency-based non-profit higher education in the U.S. We are very pleased that President Bob Mendenhall will be joining us at this year’s Learning Impact 2013 to tell the WGU story and participate in a panel on higher education leadership.

As we do with many of our keynotes at Learning Impact we have published a brief interview article with Bob.

In some respects this is sort of a “coming out” for WGU in that they have been replacing proprietary integrations (those very popular “open APIs” that every vendor likes to promote) with open standards-based integrations using IMS LTI. As the article mentions, WGU has quietly replaced 20 such custom integrations with LTI over the past several years, with probably 30 or so more to go!

Which brings us to the very critical link between competency-based learning and open interoperability standards.  The reason why WGU has so many different applications to integrate is because the best resources in different fields come from different providers.  You might think this circumstance is unique for WGU. It is not even today, but less so in the future. That’s because your departments and faculty want to use these sort of resources – or will be wanting to – and, if they are doing so now they are probably doing it WITHOUT INSTITUTIONAL INTEGRATION AND SUPPORT.  Sorry to get loud there, but frankly we are finding that many educational CIOs need to be woken up to both the challenges and opportunities (for better service) to departments and faculty. Well, in a nutshell, IMS standards are all about enabling this – just as is happening at WGU.

There is more information on how to join this open digital innovation revolution, including two special programs to aid higher ed involvement/adoption (called THESIS) and K-12 involvement/adoption (called I3LC).  

Tags:

We hope so! And we hope many universities, school districts and suppliers will collaborate on developing it!

See today’s announcement about the launch of a new collaboration to do just that.

Well, we know that the higher ed market seems to want to keep talking about the LMS, last week’s announcement from MIT and Stanford not withstanding. But, some of us are moving on. For those of us that have been attending Learning Impact the last several years (and, yes, don’t forget to sign up right now for this year’s because space is getting short!), we already know what the future of the “LMS” is (and that the term LMS is a bad name for what it has been or what it will be).  We also know what the general roadmap for digital learning resources is and how this evolution is intertwined with the evolution of the LMS. That’s because the LMS is evolving into a disaggregation of features and resources that come together easily and seamlessly for the needs of teachers and students.

The last few years have popularized, in the consumer world, the app store model. The app store in the consumer context is as much, or more, about controlling purchasing paths and revenue distribution as it is about software that the user interacts with (like iTunes). I have about eight Apple computers in my home and have been a user of Apple since the Lisa. What a stroke of genius Steve Jobs had in envisioning Apple computers at the center of home entertainment/personal digital lifestyle! And, iTunes was the delivery mechanism to make getting the digital resources easy. And, as we know, the 1-click buying, downloading, installing experience has evolved from computer to mobile devices of all shapes and sizes. Hooray!

Success of this model has lead to a lot of imitation by other large consumer-oriented companies and creating similarly vertically integrated buying experiences. To succeed at this you’ve got to have a massive point of sale presence. Amazon became the leader in e-Books. And, Google has the primary competitor to iTunes for mobile devices.

Ease of use/convenience in getting digital resources, evolving to the very popular apps (software applications) has made these vertical stores very appealing. Problem is that they also tend to lock the buyer in to a specific device or family of devices. If I want to switch from iPhone to Samsung Galaxy Note II – which I recently did – I have to start over again with the apps (Yes, the Galaxy Note II is a much better phone than the iPhone - sorry Apple!).

Of course, there are now mobile applications focused on the education segment: as our friend Robbie Kendall Melton from Tennessee Board of Regents has probably the best collection! Problem is that these vertical app stores have created a nightmare for teachers and students who generally need something that cut across many different types of devices (think BYOT). And, in order to make the user experience seamless and productive, educational apps typically require exchange of information (think user data and/or analytics) with other software in the educational enterprise (yes, like an LMS or whatever the LMS evolves to).

So, IMS finds ourselves in an interesting position in that we are going to need to enable a model in education that is not Apple, Google, Amazon (or any proprietary vertical marketplace approach) centric.  The app store project is, at it’s beginning, a collection of universities that are working to define and build a reference implementation of an app store based on open standards, that any content provider can participate in.  The advantage of building apps that utilize the open standards (think APIs – but vendor neutral) is that they will be easily integrated into a seamless teacher and student experience (yes, think 1-click). Now, will it be a gigantic app store with zillions of resources? Probably a smaller set of resources that are much more manageable for each course (while some are in love with the “learning objects in the sky” concept it is not what most faculty have time for).

The IMS educational app store project is in a top-level design phase now – with the expectation that there will be mock-ups and wireframes to discuss at the upcoming Learning Impact. From there we will herd the cats and begin building. The idea is NOT that IMS would maintain some sort of app store.  The idea is that institutions and/or suppliers will collaborate as they see fit in providing institutional or supplier-specific versions that may or may not be coordinated with peer implementations. Contact us if you'd like to get involved.

Briefly back to Apple, Amazon, Google – sorry to have to pick on you guys. But, for education it’s time to move to the next logical phase of the app store concept. The good news is that you can utilize the open app store APIs (or others can) to link the proprietary applications built for your stores into the open educational app store should the educational community wish to do so. It would be much nicer if you would spend a little bit of time and effort to engage or even contribute to the project – but we realize you are very busy making money with your vertical platform strategies and probably won’t help out the education segment.

IMSappstore

Tags:

Answer: Yes you can. Really dumb.

For evidence see Wired Campus: Stanford U. and edX Will Jointly Build Open-Source Software to Deliver MOOCs

We usually try to show restraint, but every once in a while something comes across the bow that is so completely stupid - well, it has to be addressed.

Here is what I posted as a comment to the above article:

This idea is completely brain-dead. Note to self as Stanford alumni: no more contributions to Stanford fund - as they obviously have plenty of money to waste.

Look - LINUX was based on Unix - which was a very well-baked operating system based on at least 10 years of wide adoption.  How do you create "the LINUX" of anything from a completely new platform?

And, oh by the way, the LMS market is moving to one of interconnected applications based on open standards, namely IMS LTI. This is the model that every one in the know has been talking about, and implementing, for the last three years. I know - I run IMS.  We've gone from 0 to over 150 certifications of open standards based products in three years and the curve is accelerating. You can see the many examples here:  http://developers.imsglobal.or...

So, the idea of building a platform that is all encompassing in terms of functionality - even if it is open source - is completely the opposite of where the market is moving.

Finally, when you can't find a business model, build something and give it away for free - and pretend you have a business model. When you're living off grants and endowments its a good strategy for delaying the inevitable: failure.  Unfortunately that money could have been used for something useful.

Tags:

An LTI-based prototype for a Student Progress Dashboard

At the last IMS quarterly meeting in February 2013 at Lone Star College in Houston we spent a lot of time on analytics. Analytics is a pretty hot topic in education these days. In fact, in HED the hype has been off the charts for about two years now. At EDUCAUSE 2011 analytics was the savior, At EDUCAUSE 2012 the hype was more muted - but still strong.

Why? Economics. Retaining one student is worth substantial dollars. Retaining many = mucho dollars. Not to mention national goals for graduating more students - which has a broader impact on any national economy as the delta in wages over a lifetime is large between degreed and non-degreed people.

One of the problems with the term analytics is that it is VERY broad. At our quarterly meeting we had a parade of companies (large & small) as well as very well-informed individuals working in the analytics field.  We learned that there are at least three levels of analytics applicable to education:

  1. Learning analytics: Data analysis that helps students improve learning outcomes.
  2. Academic/program analytics: Data analysis that provides information of what is happening in a specific program and how to plug holes or otherwise adjust.
  3. Institutional analytics: Data analysis that helps make decisions about how to improve at the institutional level.

There is also a fourth level - an even higher level at which governments might crunch numbers to understand a statewide or national level. Since we don't consider ourselves to be part of the government in IMS, this fourth level is not too interesting to us.

There are some great companies doing some great work in analytics. Companies like Oracle, Desire2Learn, LoudCloud, McGraw-Hill and Civitas Learning - all of whom presented at the IMS quarterly.

And, of course one of the things we have learned previously about domain-specific adaptive tutor/homework applications, like Pearson My-Labs, is that they can make use of data collected across many institutions.

The use of analytics to crunch, and potentially correlate, data from what might not appear to be related things, has appeal to many. For instance, one of the claims made by the CEO of Knewton at the U.S. Whitehouse Data Palooza event last year was that the Knewton product would be able to predict how well a student would do based on what they had to eat for breakfast! That sort of data would be very interesting to Frosted Mini-Wheats, as well as some parents.

Crunching large amounts of data from many sources and then figuring out which data is most useful/predictive is often referred to as making use of “Big Data.”

But, there is also “Small Data.” Small data tends to be more localized, and perhaps, immediately actionable (see non-education article on Why Small Data May be Bigger than Big Data). As Mark Milliron said at Learning Impact 2011, “Students are good with collecting data on them if it can actually help them as individuals.”  This makes a lot of sense to us at IMS.

Now, of course, data interoperability can potentially aid analytics because agreed upon data definitions used across many tools/products should be easier to compare. Analytics is a really important focus area for IMS - and will be a key focus at this year's Learning Impact 2013 conference May 13-16 in San Diego.

The sort of “holy grail” of data interoperability is an agreed upon “learning/progress map” that all tools and assessments could populate. Some are working on that very issue today (see for instance the Dynamic Learning Maps collaborative that is participating in IMS via CETE at University of Kansas). However, while it is relatively straightforward to agree on some types of data – like for instance assessment item results data as in QTI/APIP or usage data on things like e-books – the state of the market is that student learning models and data is in its infancy. Therefore, many tools will be producing analytics information that makes sense within the tool, but not more generally. IMS wants to put in place standards that encourage that type of innovation through variability, as well as the type of standards that capture things everyone can agree on.

To enable more use of small data in education, it occurred to us that it would be very cool if it was easier for students or teachers to simply see all of the progress data in one place – even though the tools are all separate. What a major step forward it would be for a student to work in several tools and be able to see how their results compared. So, we decided to see if LTI could be used to enable a Student Progress Dashboard that is a mash-up of many dashboards from independent tools. We see such a dashboard as displaying the unique analytics capabilities of any tool – whether or not data definitions are agreed to – and, whether or not the tool provider is willing to share such data. We think this very simple idea is empowering and will complement the progress we are making on defining agreed data fields when we can.

And, now we have a very simple prototype to show one version of the concept – using tools that are not especially analytical in nature – but ones we had lying around.  If you go to this screencast by Stephen Vickers you will see the very first IMS-enabled Student Progress Dashboard prototype. We expect this to be a standard feature to be supported in LTI going forward and want to see lots of riffing on this in the LTI community! Let us know what you think! And, tool providers, start your engines!

Note that we may not be able to tell how well a student will perform based on what they had for breakfast, as perhaps Knewton can, but, we can perhaps make a combination of tools – tools available today – more actionable for students or teachers!

Student_Progress_Dashboard_Prototype

IMS LTI-Enabled Student Progress Dashboard Prototype

 

Tags:

Yesterday we released the IMS annual report for 2012. Knock on wood, IMS has experienced seven years of steady growth. We have been very lucky in many respects, but it has been the steady support of the members and ability to change of the IMS staff that has made this growth possible.

There have been MANY headwinds along the way. A close second in that list - second only to what I'll call "jealousy" of several other failing organizations in this space that have attempted to dissuade support of IMS by completely misrepresenting what we do and how we do it - has been a knee-jerk reaction of many organizations of being negatively predisposed to joining "yet another" organization/paying membership dues. Memberships is one of the first things to be scrutinized and cut in any well-managed organization. And, there is good reason for that. In my days at Collegis I was the one doing the cutting!

Quite frankly, in IMS we are open to any model that works. By "works" we mean that actually provides the means to enable a high quality open platform - owned by none, but implemented by all - that education needs so desperately to focus investment on great products and innovation. And, in this post I'm going to tell you why we have concluded that a membership model is the best way for all concerned to achieve this.

1. Membership dues to IMS result in something of tangible value getting created - something that if done right provides huge value to the industry. IMS is truly evaluated on its ability to effect change in a very tangible way. The membership dues to many organizations are typically providing access to some amount of networking or reports that are published - reports that typically never vetted over time to see if their advice were correct or not.

2. Because there are many/diverse members paying dues and contributing to the tangible product in #1, the resulting products are supplier neutral and bring value to an entire industry (note: how the organization is set up in terms of participation, work activities and approval is very important factor in achieving this). Note that a foundation or grant sponsored project does not provide the same sort of neutral product - regardless of how the work is licensed.

3. Members of substantial numbers will only support an entity that is providing a fair and neutral process to not only create standards, but to evolve and maintain them for the changing needs of the marketplace over time. Sooooo many business models are not sustainable. Membership models, assuming that the organization is responsive, can be.

4. Commitment to the standards and the process is critically important. Membership, if done correctly, brings commitment. The leadership of the organization are committed to serve the members and the members are committed to the joint work product.

5. Stakeholders of different types and sizes get an equal vote. If the membership organization is set up appropriately institutions, government entities and suppliers can work on equal footing even though they may be paying very different membership dues. They each get one vote regardless of what they are paying, thus allowing for broad participation and a high quality product.

6. Requiring skin in the game is a good thing. It is very difficult to achieve any of the prior five things without a true commitment from true industry leaders. Requiring payment of membership dues from participants that can (special accommodation can always be made for participants who might be under financial hardship), along with a legal membership agreement to adhere to the member-approved processes, is absolutely the best way to ensure that all the activity will result in actual market movement.

7. Commitment from members results in better ways to do things. IMS has made a very large number of changes over the last seven years. Just about all of those changes were ideas from the members. I'd like to tell you that we are organizational geniuses here at IMS and came up with all this ourselves.  But, the truth is that we just tried ideas that the members had and have kept the ones that worked. And, because this is an organization set up to serve the members, they can decide to take this in any direction that they think makes sense - a very important consideration given that standards are evolving from paper specs to reference implementations to who knows what?

During the last seven years we seen at least a half dozen standards activities in this space attempt to start and die. There are several other organizations and initiatives that are hanging on just barely in terms of needed financial support. One of the reasons that IMS began publishing the annual report was to provide transparency into how well our model was working. When we started publishing the report we had no idea that we would have seven years of growth ahead of us, and, there is no guarantee of growth from here.

But, if you're looking for an organization that is creating tangible change for the better and is clearly gaining momentum - and you are willing to understand that this is "not just another membership dues organization" - well, you have come to the right place! Myself and the IMS staff consider ourselves very fortunate to be able to wrk with the many individuals and organizations noted as key leaders in IMS in the later pages of the annual report.

Tags:

This post is the letter to IMS stakeholders contained in the recently released IMS Global Annual Report for 2012: Leading the Digital Innovation Revolution in Education . Join us at the Learning Impact conference, May 13-16 in San Diego to provide your leadership to the cause!

IMS_digital_revolution

Dear Supporters of IMS Global, We are pleased to present the IMS annual report for calendar and fiscal year 2012. Maturity and adoption of IMS interoperability standards reached record levels in 2012. This support from the IMS community resulted in record levels of revenue, membership and net assets, as shown in the accompanying chart.

IMS_trends

Over the last 15 years, education has adopted many “point” examples of innovation in which technology has played an enabling role. Today’s “hottest” innovation is the MOOC (Massively Open Online Course). Will it be another point innovation or, instead, an educational paradigm “shifter?” Neither is certain, but we are certain that the least risky strategy for scaling up educational opportunity while affordably improving learning outcomes is to tap into the inexorable march of technical innovation. For this strategy to succeed, institutions must be free to select innovative applications from a variety of sources and integrate the resulting foundation of new and legacy applications. The need for agile integration and interoperability has never been higher and, thus, IMS standards and certification never more strategic.

conformance_certifications_totals

The goal of the IMS work is to enable innovation through a broadly affordable, vendor-neutral open “platform” – owned by none and implemented by all. IMS open interoperability permits the systemic integration of applications from diverse sources into a seamless, agile and information-rich educational experience, within the institution and across the Internet through its partners and suppliers. MOOCs are but one current example of an innovation that draws on the leverage inherent in IMS open interoperability.

In 2012 the IMS community provided overwhelming evidence that open interoperability can provide a cost and time savings on the order of 10-100x, compared to current, widely used integration approaches. These benefits accrue to suppliers and institutions, all while enabling a more seamless, and thus more productive technology experience for students and faculty.

IMS launched the “Open Digital Innovation Revolution” campaign at EDUCAUSE 2012 to highlight that systemic change enabled by open interoperability can have revolutionary consequences in terms of the rate of adoption of innovative digital content and applications. Moving into 2013, IMS is accordingly focused on Open Digital Innovation as a core strategy upon which to build the education markets of the future. Indeed, an emboldened breed of leadership is already emerging from the IMS community of suppliers and institutions. These bold, but practical leaders seek to amplify the value proposition of an open interoperable core platform as a means to shift today’s education paradigm towards future learning needs. The IMS board and organization are grateful to these bold leaders, who are truly leading the scalable and sustainable future of education!

Rob Abel, Ed.D., Chief Executive Officer

William H. Graves, Ph.D., Chairman of the Board

Tags:

Pages