Measuring the value of training and development is a hot topic at every conference this year and the World of Learning Conference and Exhibition is no exception.
The C-Suite is rightly challenging the L&D department to prove its value to the business. Sometimes this can seem to run counter to the themes of conferences, which often talk more about trends and technology than return on investment.
So what happens when we combine one of this year’s hot topics, Social Learning, with one of our businesses biggest needs; measuring value? A lot of blank faces perhaps? Maybe not… Here’s three ways in which you can measure the value of online, social learning activities in your workplace.
The most common report you will see about the success of a social learning program is how much was viewed / contributed / commented during the learning process. This isn’t a bad judgement; in terms of sheer numbers these figures can tell a story about how many people were reached by your learning experience.
Quite often we’re looking to create a spark of inspiration with social learning – to pass on a tweet that makes people think. To hold a mentoring session that gets people reflecting on how they work. To share a video that gets people talking. And so measuring the initial impact of that moment of inspiration probably is about counting views or comments. Social learning experiences benefit from network effects – the more people ‘being’ social, the greater the value generated.
But be warned, the figures can be misleading. In one experiment where we required everyone to participate in a social learning experiences we generated more than 3,000 responses from just 30 learners over a 6-week period. But when we came to our next measure, quality, we found that more than 95% of those contributions weren’t really valuable in terms of adding to the learning on offer. That’s a lot of wasted engagement…
Whilst measuring engagement is easy to do it is also easy to criticise. When you only measure engagement you only know that people talked, you don’t know what they were talking about. Whilst that might have sparked some ideas and further work, we’ve got no real idea that conversation equalled learning. Or do we..?
Tools like Google’s Natural Language API(1) are starting to put the power of machine learning into our hands to analyse not just how much people use social tools, but what they are saying whilst they are doing it.
We’ve taken this approach to analyse the conversation in a range of social learning experiences. Working for InterContinental Hotels Group (IHG) we measured the intentional language of contributors; who was just talking, who had identified they could do something differently and who was actually trying new behaviours in the workplace and reflecting back the results:
Knowing what ‘quality’ actually represents in a learning opportunity isn’t straightforward. For an educational experience we might equate quality with critical thought. But for a workplace experience, we’re more likely to equate quality with a change in behaviour (e.g. someone has taken a piece of learning and changed behaviour as a result). How that mounts up to business results, is the holy grail.
Attributing changes in the business environment to a social learning experience is difficult because these experiences don’t happen in isolation. Seasonal variations, the fortunes of the market, the activities going on elsewhere in the business; there are a hundred reasons that a business’s bottom-line might change.
However, the case for social learning having made an impact on the bottom line is a lot greater when you follow the value chain back down through quality and engagement. If you have evidence that people were engaging on a topic, and you have further evidence that they suggested they were going to do something different – or indeed have done something different – you begin to make a compelling case for your social experience to have impacted the company.
Post-rationalising this is very hard. In order to measure change in the business we first of all have to establish a baseline from which to work. Villeroy & Boch (V&B) showed exactly this on their way to winning last years ‘Best Social & Collaborative Learning’ award from the Learning & Performance Institute.
By comparing like-for-like sales results in a variety of global regions and using an A/B approach (some regions took part in the social experience, others didn’t), V&B were able to demonstrate the impact of social learning on bottom-line performance, taking out seasonal variations in-demand:
Getting this data took discipline and planning. Social learning experiences were heavily targeted towards specific strategic outcomes; this wasn’t a case of just letting people go nuts on a forum, but of encouraging staff discussion on particular content topics. And because V&B were also measuring the quantity and quality of the conversation as they went, the case for social learning having truly made a bottom-line difference was well and truly made.
There are literally hundreds of technologies that can help you with your measurement activities. Something as simple as Google Analytics can do a huge amount of heavy lifting for engagement metrics with just a few clicks to install.
We’re increasingly using the Experience API (xAPI), a new approach for tracking experience data in learning activities, as a key part of our toolkit. Be on the lookout at the World of Learning for companies and case studies that put traditionally hard to measure metrics into their projects.
It is entirely possible to generate credible metrics for every stage of a social learning experience; don’t let anyone tell you social isn’t measurable!
(1) https://cloud.google.com/natural-language/
Ben Betts is CEO at HT2 Labs – the R&D Company for L&D who are specialists in social learning and learning analytics/xAPI. He will be delivering a session on Harnessing the power of social learning on Day 1 of the World of Learning Conference (Session A5).