How to measure the success of your immersive learning

When it comes to measuring the success of your AR-enhanced learning solutions, it really comes down to one thing: defining clear objectives from the very start. It is crucial to define clear objectives and establish the right Key Performance Indicators (KPIs) to measure the success of your program. By aligning the KPIs with the learning goals and leveraging the capabilities of Augmented Reality (AR), you can ensure that your success metrics accurately reflect the effectiveness and impact of the learning experience.

Setting your immersive learning objectives

When rolling out an AR-enhanced learning solution, L&D professionals should focus on specific business outcomes to ensure that their efforts align with wider organisational goals and offer a meaningful return on investment.

Here are 4 key business outcomes to think about when measuring the success of your immersive learning solution:

Increase in time to competency

Compare your AR learners with more traditional learning or paper-based learning to assess whether AR has improved their time to competency when learning new information. Additionally, consider evaluating the effectiveness of AR in enhancing learner engagement and motivation compared to traditional learning methods. Furthermore, explore the potential benefits of AR in promoting active learning and providing a more immersive learning experience.

Measure:

1. Measure any noticeable improvements in task performance, speed, or quality.
2. Assess the reduction in errors or mistakes after your AR training.
3. Evaluate how quickly learners can apply new knowledge or skills in their roles after undergoing AR-enhanced training compared to traditional training methods.

Retention of course information

Test learners' knowledge retention by assessing their understanding immediately after the AR experience and again after a few weeks or months. As we’ve seen from our research with NeuroInsight, memory encoding is 70% higher with AR than more traditional forms of learning. Tracking information retention will help you gauge the long-term impact of the learning program. If your learning experience focuses on teaching specific skills, measure the learners' capability to apply those skills in real-world settings.

Measure:

1. Knowledge retention over time to determine the effectiveness of the AR solution in helping learners retain information.

Reduction in training costs and improved ROI

Because Augmented Reality (AR) can significantly prolong the lifespan of your printed materials and effectively connect the physical and digital, it is crucial to assess whether the implementation of AR-enhanced learning solutions has resulted in a favourable impact on learning costs. Moreover, it is equally important to consider the potential long-term financial benefits of integrating AR technology into your learning solutions, such as reduced printing and distribution costs, enhanced accessibility, and travel costs.

Measure:

1. Reductions in training-related travel or venue hire.
2. Savings from decreased training time or resources required (e.g., fewer trainers or less printed material).
3. The financial benefits derived from the AR solution against its costs. This should encompass both direct costs (e.g. development, hardware) and indirect benefits (e.g. improved productivity, reduced errors).

Safety and Compliance Improvements

For larger organizations and industries where safety is of utmost importance, it is crucial to measure the decrease in the number of accidents or safety incidents that occur after employees receive training. This evaluation allows organizations to assess the effectiveness of the training program in improving safety outcomes and ensuring the well-being of their employees.

Measure:

1. Number of workplace injuries.
2. Reduced errors.
3. Safety and well-being of employees.

Employer brand

As an organization, it is crucial to evaluate if the adoption of cutting-edge technologies such as Augmented Reality (AR) can positively impact and enhance your employer brand. AR can deliver more personalised comms at scale that enable prospective employees to engage with company founders and learn about the company culture in a more immersive and personal way.

Measure:

1. If the organization's reputation or brand image benefits from the adoption of cutting-edge technologies like AR through qualitative surveys and polls through 3rd parties.
2. Brand sentiment over time.
3. Candidate quality and volume.

Driving wider behavioural change and culture shift

For many learning experiences, the ultimate goal is to drive behavioural change. Observe if learners are applying what they have learned through AR in their tasks or jobs. These KPIs will help you determine the practical impact of the experience on learning behaviour.
AR and immersive technologies also give you the opportunity to keep pace with trends and drive a culture of innovation within your organization. Modern learners, especially Gen-Z, have grown up with technology and expect sophisticated, interactive, and engaging learning experiences. AR and immersive technologies can cater to these evolving expectations.

Measure:

1. If there is a noticeable change in behaviour, especially in soft skills or safety training programs. 
2. The response and engagement of your younger workforce.

How to measure the impact of AR in your immersive learning solutions

Quantitative metrics

Here are some of the metrics you can track in AR using tools like Zapworks Designer when combined with popular web analytics tools like Google Analytics or Microsoft Clarity:

Scans – how many times was your AR experience scanned? This is measured once the QR code has been scanned by your camera app, or when the deep link has been tapped.

Unique users – how many individual learners accessed your experiences?

Average dwell time – on average, how long are users spending in your AR experience?

Total dwell time – the sum of the total dwell time spent by your learners in your AR experience

Conversion rates – this can be measured from start to finish or incrementally through different stages of the AR experience to calculate drop-off rates etc.

Completion rates – what % of learners got to the end of your experience?

Geography – which countries are users engaging with your AR experience the most?

Time of day – when are your learners engaging with your learning experience?

Custom events – which buttons did a user tap on, or which experiences did a user engage with the most?

Which of these metrics you use will be heavily dependent on the objectives you set out at the start of your project? Whatever AR metrics you use to track the success of your AR experience, we typically advise you to also combine the quantitative data with qualitative feedback as well.

Qualitative feedback

Where appropriate, make sure you’re also gathering qualitative feedback on your immersive learning solutions. Use surveys and feedback forms to compare your AR-enhanced learning solutions with previous non-AR versions to ensure your learners know how to engage with your AR experience and give vital feedback on the user experience.

For example:

1. Collect feedback on your immersive learning experience to gauge learner satisfaction.
2. Monitor the ease of use and any technical issues encountered.
3. Evaluate the ability of learners to access training at their convenience and in various environments.

Usability and Accessibility

Usability and accessibility are essential aspects for the success of any learning experience. Conduct a survey with learners to gather feedback on their experience using the AR application. This will help determine its ease of use and identify any issues encountered. Additionally, track any technical issues related to device compatibility, such as problems specific to certain device types or operating systems. Considering these elements will ensure that the AR learning experience is accessible to more learners and provide a smoother user experience.

Engagement = your north star metric

As you’ve seen throughout this guide AR is all about driving deeper engagement with your learners. AR can literally place people, objectives and training scenarios in the room with them (even when delivered remotely). When thinking about using AR in learning solutions, think about AR as a way to enhance and increase engagement with your learners, whether with your existing solutions or delivering completely new immersive learning experiences with AR.

💡 If you are creating your learning content using Zapworks Designer, you have the convenient option to export SCORM packages*. These packages can then be uploaded to your current Learning Management System (LMS). By doing so, you are able to analyze learner completion data and tie that back to a learner profile. This integration provides you with the opportunity to gain a comprehensive understanding of your learners' progress and performance, allowing for effective evaluation and improvement of your learning experiences.

*currently available in beta

Final thoughts

Like with any program successfully measuring the results of your immersive learning relies on setting clear objectives as well as understanding and clearly communicating what success looks like for you and your business. Don’t forget, when looking at what success looks like it's important to combine quantitative measures such as performance improvements, cost savings, and training effectiveness with qualitative insights like learner satisfaction and engagement levels, in order to get a complete picture. The future of learning is being reshaped by AR, and through considered evaluation, we can fully leverage its capabilities to create more engaging, effective, and innovative learning environments.

If you are looking to find out more and get expert insights from L&D professionals our free guide is available to download now: Enhancing Learning Experiences with Augmented Reality