The use of data and metrics is invaluable for L&D professionals. Not only does it allow us to measure the success of our learning interventions, it also enables us to better identify the needs of our learners. Before the release of xAPI 1.0 in 2013, we only really needed to consider how to deliver good e-learning on the page. Now, alongside deciding the best pedagogic fit for our learners’ needs, we must also consider how to track outcomes using the wide range of tools both within an LMS and outside of it.
What is xAPI?
xAPI (full name: The Experience API) is a software specification used in e-learning. It allows learning content and learning systems to ‘speak’ to each other in a way that records and tracks learning experiences. For L&D professionals, xAPI offers the opportunity to track more than just progress and scores. We are finally able to consider the bigger picture – from learners’ initial thoughts about the learning, through to the impact it has on their everyday working life.
Some of the most significant things xAPI enables us to do include:
- Designing more readable, qualitative and quantitative progress and score reports
- Considering more widely what should be measured as part of a learner’s learning journey and as an outcome of learning
- Capturing previously intangible behaviours that demonstrate or have an impact on learning
With xAPI, L&D professionals can now consider more widely what we want to track and how we want to track it. We can look at previously intangible activities (such as social influence, creativity etc.) and give learners credit for all of the learning and development they complete. Perhaps a more fitting title than L&D professional, therefore, is 'performance analyst'?
L&D professionals as performance analysts
When it comes to analysing the performance of your learning interventions, what measurement model should you use? There are several models out there but the most popular one is the Kirkpatrick-Phillips Model. This model considers the value of any type of learning, formal or informal, across five levels. Dr. Donald Kirkpatrick originally defined satisfaction up to results across four levels in the 1950s, then in the 1990s Dr. Jack Phillips argued for the addition of ROI as a fifth level – the cherry on the top.
Level 1: reaction
How favourable, engaging and relevant the learning is.
Level 2: learning
The specific and measurable learning outcomes gained from successful completion of the learning.
Level 3: behaviour
The extent to which learners apply their newly acquired knowledge, shown as a change in the learner's behavior that can be attributed to the learning outcomes achieved.
Level 4: results
To what extent behaviours are being displayed that contribute to specific outcomes being achieved or the specific outcomes being achieved as a result of the learning. This is often evidenced as the measure of business improvement as a result of the behavior change.
Level 5: ROI
Compares the monetary value of the business improvement with the total costs of the learning programme.
By adding that fifth level in the 1990s, Dr. Phillips recognised the importance of evaluating the impact of broader L&D activity beyond just bottom line figures.
Previous e-learning standards (such as SCORM) largely just facilitated data capture of ‘Level 2: Learning’. Using xAPI, we are able to consider impact across all five of the Kirkpatrick-Phillips levels, capturing data on a range of behaviours. Let’s take a look at a quick example.
The growth of NewWay Training
NewWay Training are a large (fictional) health and safety training provider. In addition to well-respected classroom training, they have recently begun to sell e-learning courses online. They currently have a small team creating these online courses, but would like to expand the amount of courses available to their customers.
NewWay’s senior instructional designer delivers a course called ‘Create Your Own E-Learning’. Classroom trainers who complete the course are recognised within the LMS, and are incentivised to create and publish an e-learning module of their own.
Using xAPI, NewWay Training are able to capture data on the following behaviours occurring both within their LMS and outside of it:
- User feedback of the ‘Create Your Own E-Learning’ course (Kirkpatrick-Phillips Level 1)
- Progress tracking and assessment scores for the ‘Create Your Own E-Learning’ course (Kirkpatrick-Phillips Level 2)
- Trainer develops their own e-learning module ((Kirkpatrick-Phillips Level 3)
When it comes to Levels 3, 4 and 5, xAPI allows us to capture experiences outside of the learning platform. At Level 3, we can do this by reporting using xAPI statements such as ‘Sarah published an e-learning module to the store’. At Levels 4 and 5, other business processes can be used to capture data that correlates to learning. These are experiences and contributions that indicate learning, but would have previously gone unrecorded.
Introducing... The Intangibles
Historically, these ‘intangible’ learner contributions have been difficult to capture within an LMS environment. Thanks to xAPi, we are now able to track less conventional and more informal actions indicative of learning. This is especially important when you consider the 70/20/10 Model, which suggests that:
- 70% of learning is experiential, occurring through daily tasks, challenges and practice
- 20% of learning is social, occurring with and through other people (e.g. our co-workers)
- 10% of learning is formal, occurring through structured training courses and programs
The ability to track experiential and social contributions is so vital to measuring workplace learning. If you don’t have the ability to capture it, then potentially you are only looking at 10% of the whole picture. Experiential and social learning is best supported by just-in-time learning resources, social networks and on-the-job training. For example: by using xAPI statements, we can see who views/downloads resources – whether that is documents, videos, content packages or even webpages. We can also log forum posts, acknowledge membership and contributions to social subject-specific online groups, record attendance at face-to-face events (such as discussions or meetings), and capture actions (such as organising an event, or creating a network).
How can intangible behaviours be tracked and recorded?
I’ve spoken a lot about what xAPI enables us to do as L&D professionals, but you may still be wondering exactly how you can make use of it. The answer is: through your LMS. The LMS your organisation chooses should be an effective vehicle for identifying, developing and harnessing its intangibles. It should first enable you to identify the right attitudes and behaviours, and then track, measure and develop these. Finally, and possibly most importantly, it should allow you to recognise and reward these behaviours, encouraging positive outcomes within your workplace.
An LMS that does this is our very own Thinqi, what we at CDSM like to call ‘The Blended Learning Ecosytem’. Thinqi lets you define and discover the intangible characteristics that indicate true potential in your people. It uses the xAPI specification to support you in identifying, tracking and developing the people your organisation needs for a bright and prosperous future. As L&D professionals, you are no longer limited to tracking progress and scores in a self-contained learning object. The future is here.
We're always exploring key trends in the learning and development world, so keep an eye on our blog and social media channels to see when new insights are published: