So, You Want to Win a Learning Impact Award

Contributed by: Dr. Colin Smythe, 1EdTech Chief Architect

February 2023

The Do’s and Don’ts for Winning a Learning Impact Award: Insights and Advice from a Long-time Judge

The Learning Impact program was introduced by 1EdTech in 2007. Of course, at that time, we were IMS Global Learning Consortium. The program, which includes the annual Conference and Awards competition, aims to recognize innovative and influential uses of technology to support learning and teaching worldwide. Through it, we seek to identify the repeatable usage of teaching and learning technology to help private and public institutions and educational authorities to:

• Increase access

• Create personalized learning

• Improve student engagement and experiences

• Promote actionable assessment

• Advance edtech ecosystem evolution

The program's emphasis is on...drumroll, please...improving learning impact. The focus is not on just using technology in learning. It is not even about using interoperability standards to get better information exchange. It is about improving learning and teaching experiences through the use of technology.

Award Nominations

The Learning Impact Awards process consists of three stages. The first is the initial submission through the online form.

Step 1: Application

The award nomination consists of a descriptive title, a brief (<1000 characters) overview of the project, and, most importantly, detailed descriptions (<2,500 characters) about the impact on Personalized Learning, Institutional Performance, and the Digital Learning Ecosystem each. Applicants must also categorize the maturity of the activity (Research, New, or Established) and its scope (International, Countrywide, System/School District, Institution/School, Departmental, or Workforce/Education-to-workforce). It doesn't matter who submits the nomination, but the project must involve a hosting user institution and the solution supplier.

Step 2: Supporting Materials

The finalists—typically between 30-40 organizations—are selected from the online nominations, plus the winners (top submissions) from regional competitions like the one hosted by the 1EdTech/IMS Japan Society. Finalists are selected by the 1EdTech team based on criteria to identify exemplary implementations of technology that demonstrate the greatest impact, or potential impact, on addressing the challenges facing the global learning sector and possessing the greatest potential for generating a positive return in investment. Most of the applicants rejected at this stage are because they are just a sales pitch for a particular product. While most submissions are from North American organizations, a growing number are received from the rest of the world. The second stage consists of:

• A one-page document in English outlining the challenge, solution, learning impact outcomes, and return on investment
• A four-minute or less video pitch about the project

The project materials are made available to the judges for review (and later posted on 1EdTech's website for everyone to access). The session with the judging panel is the third and final step.

Step 3: Presentation

• A ten-minute live presentation to the expert judges split into five minutes of demonstration/visuals and five minutes of question and answer. Each of the five-minute periods is rigorously enforced.

This year, the final presentations to the judges will be on Monday, June 5, at the start of the Learning Impact Conference in Anaheim, California.

Good quality information for all three phases is important. A good submission provides complementary information.

Judging

The Learning Impact Award judges are volunteers from the 1EdTech Contributing Members institutions and staff, mostly from North America. 1EdTech Members are from 28 different countries, so more than 20% of our membership is outside North America. Around the world, there are different perspectives on Learning Impact, and so we need more volunteers¹ from our non-North American institutional members.

There is no limit on the number of years someone can be a judge. I have been one since 2007, but most people do it for 1-2 years. Being a judge takes approximately 2-3 days of work over a couple of months (including the intensive day of the presentations). Depending on their preferences, judges are allocated to either the Established or the New/Research categories. We would like a balance of judges from higher education/K-12/corporate training, but we have to use those who volunteer. Typically, each judge gets 15-20 entries to evaluate (we try to keep the number of successful submissions in each category balanced).

Each judge has their style of preparation, but everyone has done the background work before the day of the presentations. My approach is to read all the online submissions to get a feel for the entire set I’m judging first. I then go through each submission and read the 1-page document and the accompanying video. Only then do I go back and allocate my initial scores. Each judge must score each submission on the evidence for the Impact on Personalized Learning, Impact on Institutional Performance, and Impact of the Digital Learning Ecosystem. We use a scale of 1-5, with 5 being the top mark. The final step is to listen to the presentations and raise any questions. After each presentation, I amend my scores; it is not unusual for at least one score to be changed. Each judge scores independently, and there is no discussion between the judges about their scores.

Scores and Awards

The final scores are produced by averaging out the scores from judges. One final tweak is that we provide an opportunity for everyone to rate the submissions (although they do not attend or see recordings of the presentations and Q&A). The scores from the public voting are averaged to become the equivalent of one other judge. We then take the ordered list and name two winners for each award level: platinum, gold, silver, and bronze.

There is no balancing of winners with respect to the types or maturity levels of the project categories. Occasionally, but rarely, when the scores are very close, we can have three winners in one award grouping. Winners are announced during one of the general sessions at the Learning Impact conference and in a public press release.

What Makes a Winner

The five features that are most common in platinum, gold, silver, or bronze awards winners are:

1. Provision of clear, strong evidence that supports the claims being made in the application for the improvement in learning impact. The objective is better than the subjective. Facts-and figures are more important than verbal supporting statements.

2. There are clear and significant benefits for each of the Personalized Learning, Institutional Performance, and Digital Learning Ecosystem perspectives. That does not mean that the benefits are equal in the three perspectives.

3. The focus of the information presented in the three formats is on the improvement in the learning impact. It is clear how the new technology is being used to overcome a weakness or provide a new teaching and learning capability that was otherwise missing.

4. Solutions make use of the relevant technical standards and specifications. This includes, where appropriate, the use of the 1EdTech specifications (it is a "horses for courses" approach, but the judges are skeptical of any solution that uses no technology standards or specifications). Technology solutions that focus on providing a total edtech solution invariably do badly. The aim should be to enable combinations of innovative solutions and strategies and to avoid ‘lock-in’ to any product and/or vendor.

5. There is a clear and strong working relationship between the submitting supplier and the institution. Furthermore, the more submissions an organization makes to the same Learning Impact Award competition, the less successful the outcomes. Organizations with similar submissions over multiple years are also less successful. Quality, focus, and clarity are key.

What to Avoid

The five most common mistakes, in order of significance, made in submissions that are not platinum, gold, silver, or bronze award winners are:

1. The improvement in learning impact is unclear. Remember, this is an award for learning impact, so be very clear on how learning and teaching are improved. This is not just about the use of educational technology. It is about using such technology to improve teaching and learning experiences.

2. There is a lack of information or participation from the institution (particularly during the presentation and Q&A session) that benefits from the learning impact. This is supposed to be a joint submission from the supplier and institution using the technology.

3. Insufficient context is supplied. This is very important for non-North American submissions. What may be mundane in North America (and remember, most judges are North American) may be state-of-the-art elsewhere. It is about the improvement in learning impact. Do not assume the judges will appreciate the learning and teaching context in your country; they will probably not.

4. The same information is repeated in the same way in each of the three submission formats (one-page document, video, and presentation). The video is a great way to demonstrate some key activities. I would avoid too many talking head clips. The one-pager and presentation are great for summarizing facts and figures;

5. The timing of the presentation goes awry, and key points are not covered before the end of the strict five minutes. Get the key points presented early so that nothing important is lost if you run out of time. Of course, rehearsal of the five minutes is important.

Like most successful endeavors, early planning is important. When preparing your submission, make sure you provide:

• Evidence, as much as possible, for the claimed improvement in learning impact
• Information for all three evaluation criteria (Personalized Learning, Institutional Performance, and Digital Learning Ecosystem)
• Evidence of strong collaboration between the solution supplier and the hosting institution
• Complementary styles of information using the different submission formats

The nomination period for the 2023 Learning Impact Awards begins on January 31 and ends on February 28. Click here to submit. Good luck!

¹If you are from a 1EdTech Contributing Member district, state, or higher ed institution and would like to volunteer to be a judge at an upcoming competition, contact Cara Jenkins at marketing@1edtech.org.

1EdTech HED Talk

Contributed by:

Andrea Deau, Senior Director for Higher Education Programs, 1EdTech

Overcoming “Data Blindness”

New research published in the peer-reviewed Journal “Science” found that despite the vast amount of data available to universities, they lag behind industry, business, and government in deriving strategic value from that data. It can be difficult to achieve your strategic goals if you don’t know what you’re working with or if you can’t track your progress.

The study points out that one issue with universities being “data blind” is that many times, there is no one in charge of the data, and when there is, they have so many other responsibilities that it can fall by the wayside.

In my years of working at a research university and now with various higher ed institutions, I know there are a few steps institutions can take to make collecting and effectively analyzing data easier and more secure.

One of the big concerns is data privacy, which can get complicated with evolving rules and regulations. 1EdTech does review the privacy policies of thousands of tools and applications, and you can find more information on that in our TrustEd Apps Directory. We also have a TrustEd Apps Management Suite currently being used by K-12 districts, which lists curated edtech apps validated for privacy and integration. We are working with our members to make it more applicable to higher ed this year.

Outside of that, much of the work is already done for you, and many of you may already have or are currently creating a digital ecosystem for your campus. As you build it, I recommend using interoperability standards, including those created by the members of and certified by 1EdTech, to ensure your technology, tools, and apps all work seamlessly together.

An interoperable system also makes data easier to share between tools and systems so you can collect the information. Once you have it, standards, including 1EdTech’s Caliper Analytics^®, can help make sense of the data and ensure you’re comparing apples to apples.

We are already seeing success with this strategy through our 1EdTech members.

The University of Michigan created the My Learning Analytics (MyLA) tool to provide students with information about their course engagement, which helps guide their decisions to improve academic outcomes and set personal goals for individual courses.

“Supporting faculty innovation is a cornerstone of our work, which means we need to be able to support a wide variety of edtech tools while also creating our own tools that enable new possibilities,” said Sean DeMonner, information systems executive director of teaching and learning at the University of Michigan. “The connectivity provided by 1EdTech standards, as well as our ability to influence those standards, means that we can enable seamless, data-rich integrations of commercial tools and our tools at a fraction of the time and cost while also trusting we can support a much larger collection of options for the faculty.”

Thanks to its data management platform, the University of California San Diego knows when it may have a student who needs additional support. The Student Activity Hub provides one secure space to store learner data without needing to pull the information from different applications. That allows faculty and advisors to track student progress and connect with student-facing technology.

“Our student activity hub helps us improve our students’ success, including enabling broader and deeper insight into student progress, personal messaging and reminders, understanding the impact of co-curriculars, and nearly limitless possibilities on connecting data from our edtech ecosystem,” said Vince Kellen, chief information officer at the University of California San Diego.

Penn State is consistently working to evaluate and, when needed, improve courses to serve students better. The university’s course improvement model asks faculty to evaluate and refresh courses on a regular basis. To do that effectively, they need data—especially data that tells them what content is being used and how.

“Having data that's easy to access and review is powerful when making decisions about tools and content to include in a course,” said Jen Stedelin, vice president for information technology and chief information officer at Penn State. “We no longer have to rely solely on anecdotal evidence. Data also helps us better understand student behavior, so we can reach out and offer more timely help to students who may be struggling.”

These are just a few examples of 1EdTech members leveraging interoperability and standards to move away from “data blindness” and instead use the resources available to them to achieve their strategic goals.

About the Author
Andrea Deau is the senior director for higher education partnerships. She works closely with members to meet their challenges in the rapidly growing and evolving digital teaching and learning landscape. She has a robust professional history as an academic innovation leader focused on student-centered education and experiences.

1EdTech Talk on Credentials

Co-Authored by:

Jim Ireland, Executive Director, HR Open Standards Consortium

Mark Leuba, Vice President of Product Management, 1EdTech

Bridging the Gap Between Education and Employment

Imagine this, an employer wants to find someone with a specific set of skills, and they can sort through applicants based on those specific skills with just a click of a button. Not only that but applicants’ verified skills and accomplishments are highlighted, regardless of when or where they earned their credentials. It’s a world where employers find the people they need, and applicants, who may have been ignored at one point, are judged by what they know, what they can do, and nothing else.

It sounds idealistic, but that world is not as far away or as impossible as you may think. We can already see examples of that working in small pockets around the world, and with the new collaboration between 1EdTech and HR Open, we believe we can take that vision worldwide.

How? By using open standards to make it easy for everyone involved. It’s what 1EdTech and HR Open do best.

1EdTech’s Comprehensive Learner Record (CLR Standard™) and Open Badges create digital records of a learner’s or worker’s achievements that they have agency over and can share with potential employers. Through 1EdTech standards, these credentials are easily verifiable, so an employer doesn’t have to guess about the authenticity or what a specific credential means. Instead, they know exactly what the applicant brings to the table and can automatically verify it.

Still, these verifiable digital credentials are only useful if employer application systems can read them. That’s where HR Open Consortium comes in, the only independent, non-profit, volunteer-led organization dedicated to developing and promoting a standard suite of specifications to enable human resource-related data exchanges.

By creating a standard digital resume, an applicant can input it into the system, and the employer receives the pertinent information they need. The new Resume CV standard aims to include the ability to transfer credentials from a digital wallet, increasing the quality of information they receive from the resume.

The whole process can work if the technology speaks one language, and standards allow them to do that. Without standards, it’s like trying to charge an Android phone with an Apple charger—it’s not going to work.

Like we said, this is already working in small areas, where institutions of higher education, K-12 districts, and local businesses work together to create a system of developing and sharing credentials. You can read about a few examples here. The goal is to make this work on a larger scale. That’s why 1EdTech’s CLR is recommended by AACRAO, and we’re working to align our standards with W3C to help get everyone on the same page.

By combining the standards in education (1EdTech) and employment (HR Open), we hope to bridge the final gap to more efficient, effective, and equitable hiring practices.

There is still a lot of work to do, but we’re getting there. We hope you’ll join us at the next Digital Credentials Summit in Dallas, Texas, February 27 – March 1, 2023.

You can also participate in our monthly Digital Credential Roundtables and view past conversations online to learn more about this exciting work.

About the Authors

Jim Ireland is the executive leader of the HR Open Standards Consortium, an independent, non-profit, volunteer-led organization dedicated to developing and promoting a standard suite of specifications to enable human resource-related data exchanges.

Mark Leuba is a technology leader in education with particular expertise in online and competency-based education. Mark's role in 1EdTech is to guide its product management strategy and team, building on the substantial success of the 1EdTech founding team.

1EdTech Talk on Credentials

Contributed by:

Kelly Hoyland, Higher Education Program Manager, 1EdTech

Creating a National LER: How 1EdTech Can Help Get Us There

The debate over how or if higher education needs to evolve always comes back to a few central needs and expectations of the students. To sum up, higher education is expensive, so people want assurances that their large investment will get them started on a rewarding career path once they graduate. That concern is compounded for people who aren’t certain higher education is the right path for them because if they don’t finish the degree, they lose all of the time and money they invested before leaving the institution.

One promising solution to all of these challenges, and one that is getting more and more attention, is Learner and Employment Records (LER).

LERs are records that individuals can hold, control, and build on throughout their lives as they acquire new skills. This includes everything from credits earned in higher education to professional development programs, certification assessments, and more. Thus, giving learners the ability to see how their investments tie into career requirements, maintain credit for their work even if they don’t complete a full degree, or can finish the degree on their own time. This also helps improve equity by providing flexibility in how and when a person earns the credentials they need for the career they want.

A recent whitepaper from Central New Mexico Community College, IBM, Western Governors University, Randa, Public Consulting Group, and Solutions for Information Design lays out recommendations to create a successful national LER ecosystem. Several of the contributors are 1EdTech members, and if you have the time, it’s worth a read, but here are my takeaways on how 1EdTech can help you get started with digital credentials and begin creating a national LER ecosystem.

1EdTech, our members, and partner organizations are already working on several of these recommendations. So, while it may feel overwhelming, starting your digital credential program from scratch is no longer necessary. Rather, a large community is ready and willing to help get you where you need to be, so you can help us make the dream of a national system a reality.

Breaking Down the Recommendations

So why are these steps so important? Let’s take it one at a time.

First, creating LER standards to meet the needs of the job market.

As the job market and the needs of employers evolve, so does the need for workers to reskill and upskill throughout their careers. Not everyone can return to school whenever a career change is needed. Instead, employers need a system that connects them to people with specific skills earned in various ways.

That system needs to be equitable and efficient to work, and that is where standards come in. The standards help make any credential work in any system. I like to use electrical outlets as an example. You can plug any lamp (credentials) into an electrical outlet (hiring system) and trust it will work because both the lamp and outlet are built to specific standards that ensure they will work together. It doesn’t matter if the lamp is tall, short, bright, changes colors, has a shade, or is a bare bulb, it will work in any outlet because it meets standards. All you have to do is pick the lamp you like and plug it in. That’s what standards do for credentials, you pick the ones you want, plug them into the system, and you can see what each applicant brings to the table without worrying if the format their credentials are in will work.

This does require all stakeholders to agree on one set of standards, which is why 1EdTech looks to partner with and align standards with other organizations, including aligning the latest Open Badge and CLR standards with W3C. This helps ensure we are all moving forward together. 

Second, create a technical infrastructure that provides shared identity/trust and skill/credential services.

Seeing what skills a person has is one thing, but knowing what those various credentials and accomplishments mean is something entirely different. 1EdTech’s Comprehensive Learner Record (CLR) standard can help with that too.

The study points out that the increasing diversity of job classifications and new job categories make a national LER system even more important. Employers are looking for individuals with specific skills, and they need to understand what each degree or credential means because they can mean completely different things depending on how a person earned them.

1EdTech’s Open Badges and CASE standards help serve as a type of decoder ring, providing context and data for each credential so the person reading them can verify the skills match what they need and that the credential comes from a trusted source.

This is important because currently, credentials are shared by the institution or company that issues them. So while they are trustworthy, the person who earned them can’t necessarily control or share them. Open Badges and CLR give the earner agency over their own credentials while also providing verification of validity.

Third, integrate talent marketplace provider offerings with the LER infrastructure.

Once you have the credentials, HR professionals and academic institutions must be able and willing to use them. 1EdTech’s Wellspring Project is focused on bringing institutions and employers together to drive the use of digital credentials and automate matching credentials with educational and employment opportunities. We also help facilitate conversations between educators and employers to ensure that what is taught in the classroom matches the skills needed to find a job.

To help move this forward, we recently partnered with the HR Open Standards Consortium to match digital credential standards with resume standards, making them easier for employers to access and use.

Fourth, invest in regional LER projects.

This isn’t something 1EdTech can do directly, but our contributions to the first three recommendations allow our members and organizational partners to take this need and run with it. The white paper mentions several case studies of their partners and 1EdTech members creating valuable and verifiable digital credentials in their local ecosystems. You can find more examples on 1EdTech’s Achievement, Opportunity, and Employment Imperative web page. These organizations are leading the way in creating a national LER ecosystem by proving it can work on a smaller scale.

The fifth and sixth requirements are creating legal and regulatory terms and creating an organization that certifies LER technologies and applications.

These are beyond 1EdTech’s purview, but we continue to partner and work alongside groups taking on these important issues to ensure the standards support and evolve along with the needs of learners, employers, institutions, and industries.

It’s a collaborative effort, one that is close to 1EdTech and our members. We are collaborating to create an open, trusted, and innovative ecosystem that works to benefit all. That’s what we do here. We join forces and find solutions together because we know that for this to succeed, it needs to work for everyone.

About the Author
Kelly Hoyland serves as the program manager for higher education at 1EdTech, where she works with members to meet the challenges they face in the rapidly growing and evolving digital teaching and learning landscape. This includes working across K-12, higher education, and corporate education to make life achievements more accessible, personalized and equitable from the start for every learner.