Learning Analytics Methodology

Like This Article? Share It!

Most of the time, complex tasks are best approached in bite-sized pieces. In our 20 plus year history, Riptide Software has developed and delivered complex distributed software, managed implementations, and sustainment of these systems.

It is very common for us to use Crawl, Walk, and Run in our proposal language and as we build the schedules for these programs. The modernization of complex enterprise ecosystems can be accomplished by incrementally building to the level of complexity that becomes the final outcome. At Riptide, this is how we work. It’s 2018 and the opportunity has risen for a fresh start in many organizations and departments. You might have an exciting laundry list of tasks on your organizational road map to be tackled this year, some being more time-consuming and complex than others. Maybe learning analytics and xAPI is on your roadmap? If so, you know that this can either be very simple, or complex, depending on what exactly your goals for this technology are.

Want a visual? Check out our Crawl, Walk, Run Infographic!

We’ve all been put into the situation at some point in our lives of taking on a large project, rushing through it or not having clear direction, and coming out with a less-than-satisfactory outcome. For example, anyone remember rushing through group-projects the night before the deadline during college? Maybe this wasn’t you, but many professionals in L&D work in teams or cross-departmentally to get these technological adoptions accomplished and this is where you know that the importance of a clear process can be appreciated. Simply put, you have to know how to crawl before you can begin to walk, and you have to know how to walk before you can effectively run. It is comforting to know, by following this methodology, that you can start crawling now – even if you don’t feel 100% “ready” to take on the full-breadth of learning analytics quite yet.

This is by no means comprehensive, but we’ll briefly walk through these 3 steps that have helped provide successful results for many of our clients:

Crawl

The “Crawl” step  is the simplest path where you can prioritize and identify goals you’d like to accomplish that may or may not work towards a more complex outcome. In the case of xAPI and learning analytics, we like to have our clients identify the most important training initiative they can think of that they’d like to get more information out of. We start by connecting to this one piece of content, setting up a data visualization and reporting tool, and showing our client the data and reports they can now get. Sometimes crawling takes the form of a Proof of Principle (PoP) which might be connecting to one important system/piece of content, or two, and proving that the technology can effectively integrate across multiple systems. This stage is a great point in time for you to champion the technology by showing results that will be directly responsible in helping solve tangible business problems.

Riptide Elements Crawl

A simple example of “Crawl” might be getting additional xAPI behavioral data out of the LMS. This data could then be reported on through data visualizations and catered to different audiences to be shared with stakeholders, course developers, and even the learners.

One of our recent projects that encompasses the crawling stage well is a client who was simply looking to get more data out of an assessment within their LMS. Before the project began, the client was getting no data out of this one assessment – it was impossible for them to understand what their learners were doing or answering on the module. We quickly hooked them up with xAPI-powered analytics that reported to a Learning Record Store (LRS) and ended up with a very excited and happy client. They were now getting all sorts of data they were missing out on including capturing the text from the questions and answers themselves in the LRS. During a recent follow-up call, one of the instructional designers was even building out their own reports and showing the rest of the team the trends they were discovering based off of their newfound data. A crawl can even be large in scale if it is not complex. For instance, if you can add a single xAPI profile/LRS endpoint to hundreds of similar learning content packages, you could get all of those lessons reporting a large amount of data in one simple step.

Walk

During the walk stage, we get a little more complex. It could be in the types of systems we are connecting to or the sources of data we are reporting on. This might involve enabling an entire software program with xAPI technology to get all kinds of learning analytics data about consumer/learner usage. If you started with crawl, you’ve already proven the concept of the work. Now you connect, prioritize, and add on systems as necessary to get the types of data and insight you are after. Here is where you’ll also be able to build better reports and correlate data in a data visualization tool or have the data sent to your own Business Intelligence tool that you’re comfortable with.

Riptide Elements Walk

A simple walk example might be connecting to more complex systems where you’d like to get more behavioral, xAPI activity streams from.

A good example of a client going through the walk stage is an authoring tool company. This client offers a very robust authoring tool and as a simple crawl, they use our Waypoints productto teach new users how to use their authoring interface. The client also wants to understand what features were being used, and what basic functions should be suggested as people were using the authoring tool. As a “walk” step adding more complexity to their implementation, this client has plans to connect the usage of their software platform to our Storepoints Learning Record Store, which will allow them to better understand how to teach people about their product.

Run

Running is the ultimate stage and frankly, if you want to hit the ground running, it is most important to gather proper requirements, analysis, and planning. Some clients stop and are perfectly happy at the “Crawl” stage. Others stop at the “Walk” stage and don’t have a desire to go further – that’s perfectly fine! This is the stage where we get into adaptive and automated techniques where we (or our clients) are building rules and workflows around the types of training being delivered. This is also the stage that our clients are using data visualizations and reports to evaluate the training (what works, what doesn’t, and why) and adapt to it based off of the data they are getting and what trends are showing.

Riptide Elements Run

A simple run example would include connecting all of the systems you’d like to gather data from, and ultimately, mapping these activities/training to on-the-job performance, showing how these activities affect specific KPI’s, and adapting your L&D strategy to account for these trends.

A very sophisticated client of ours is a great example of the Run phase. This client sells a revenue projecting software program for the hospitality industry. The software program is completely enabled with xAPI technology, is reporting data from multiple sources, and it allows the hospitality clients to accurately project revenue for the next quarter with a 97% accuracy based on numerous external and internal variables and historical data – amazing stuff! This empowering data allows these clients to take a look at these variables, and determine what can be tweaked in preparation for the next quarter to help them increase their revenues.

In addition to this, we have a couple of clients who are learning content creators. They produce a distance learning product that they sell to multiple enterprise companies and they are able to gain insights and data from their product no matter where it is being hosted. One of these clients has been with us for 5 years and during large training events we have gigabytes of data per second coming into the Storepoints LRS from all over the world.

You Can Start Simple…

The crawl phase is about simplicity. We recommend that all of our new prospects just getting into learning analytics and xAPI go through a crawl phase with us. This allows us to show you that the technology works as intended and gets you the data and reports you’re looking for. It’s easy to get started, and we’ll walk you through each step of the way to get to the run phase if you so desire.

Remember as you plan your 2018 calendar year, the time to start crawling is now! The amount of data and analytics you’ll have by the end of 2018 will allow you to craft insights at the end of the year and plan for a successful, and impactful 2019.

 

We’d love to hear about your learning analytics goals: Contact Us to Preview Our Capabilities!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>