big data for learning and development

Like This Article? Share It!

We love big data – especially for the Learning function. You may have wondered what big data entails, and how it can aid you in your day-to-day job. Or maybe you’re already using it! This post is dedicated to introducing big data for learning and development.

Unfortunately, big data has become a popular buzz word within many industries and therefore sometimes used incorrectly. If you’ve never worked with big data, it can be an intimidating concept and not especially easy to understand. We’ll discuss a high level view of big data, as well as the outcomes that can be achieved once you implement an xAPI big data-friendly strategy.

Unfortunately, big data has become a popular buzz word within many industries and therefore sometimes used incorrectly.

Simplifying Big Data

Big data can be broadly described as very large sets of data. This data is so large that it can only be analyzed through sophisticated software programs to reveal trends, patterns, and associations that relate to human behavior and interactions. In order for big data to really be, well, big data, it needs to fall within 5 dimensions, or attributes:

Volume: This is the quantitative scale of the data. Most US companies have at least 100,000 Gigabytes of data stored. When it comes to data volume, it’s not only important to capture the data, but to be able to filter relevant data as well. Implementing the right software, such as Storepoints LRS, to help filter through this data is extremely paramount here.

Variety: Variety covers structured and unstructured data sets as well as data sources. Structured data follows a neat set of rules (such as date, time, or currency), while unstructured data (such as a voice message, photograph, or social media post) may not. Sources of learning data may be an HRIS system, LMS, Sharepoint, or website such as or Youtube.

Velocity: Velocity is the frequency of data being collected that needs processing. A good example of velocity is thinking of the massive amount of credit card or stock transactions happening worldwide every minute of every day.

Validity: This covers how reliable your data is. Can you trust that your data is trustworthy and representative of what you’re capturing? Not all data is useful. A good visualization and reporting tool can help you weed out what’s important.

Value: This should be the most obvious – is your data beneficial and useful? You should be able to harness insights from the data you’ve collected to answer questions, inform decisions, solve (or even discover) a business problem.

Outcomes of a Big Data Strategy

We’ve just scratched the surface when it comes to understanding big data, so what are the potential outcomes for L&D? Good news – they are limitless! Some of our clients are extremely sophisticated and currently gaining amazing insights from their big data and systems integration done by Riptide. The outcomes range from:

  • Understanding Courses: Using big data to create data visualizations for engagement metrics on a training module and assessments (scores, time spent per page, time spent per video, time spent per question, when the user scrolled through video, when user paused video, etc.)
  • Training Soldiers to Shoot: Using big data to train soldiers how to shoot weapons on training ranges through automation. Based on the soldier’s data gathered from systems such as the gun scope, the targets, length of lanes, etc. training is automated to tell soldiers that they may have a breathing problem or proper aim issue.
  • Forecasting Revenue: A client uses big data for their software product which forecasts their client’s future revenue based on massive data sets for up to a 99% accuracy.

We simplify big data by capturing all of the data in a data specification called the Experience API (xAPI). This turns any data into a structured format (Remember “Variety”?) which makes it easy to read and process by computers and humans!

We are now at the point in Learning and Development where an individual’s characteristics, previous job experience, professional development and training data can be merged with real-time, work-related data that can give a company opportunities to make data-driven decisions about who to hire, ways to improve sales performance, how to reduce overhead costs, how to increase productivity, and so much more.


If you have any questions about connecting disconnected systems and getting all of the data – get in touch! We’d love to help.


Trying to get buy-in for xAPI and big data learning analytics for your organization? Check out our 1 or 2 day come-to-you workshop:

xAPI Workshop


  1. Sundaresh K A

    big data has become a popular buzz word within many industries.most of companies required bigdata expert. thanks for sharing useful content
    Bigdata training courses

    • Christy Puller

      Thanks for your feedback! We are seeing data scientists becoming needed as enterprise organizations expand into the big data space.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

This site uses Akismet to reduce spam. Learn how your comment data is processed.