To what depth do you track user’s abilities to use a software, or learn a skill in digital learning? Some organizations track completions and/or scores to “check the box” on a competency or skill.
Recently, competency based training has become a topic of interest in talent development. Competency based training applications can be methods of instruction, assessments, grading, and evaluation/reporting techniques that can accurately prove students have demonstrated knowledge or skills. It’s now easier than ever to implement – here are a few steps that can get you started:
Stand on the Shoulders of Giants
This step involved utilizing the Experience API (xAPI) technology. xAPI is based on “Activity Stream” specification work that was done in collaboration between Google, Facebook, Microsoft, and others, to track digital experiences in a standard format. These activity streams can add context and meaning to learning and development that is not as easily available by other means today. To date, approaches to competency-based training applications involve proprietary data transformations, challenging and large-scale design processes and other obstacles to evaluate how and if competency has been attained. Today, talent development has begun adopting xAPI solutions as an enabling technology, which allows for a simple and interoperable way to address and evaluate competencies and map training to performance data.
Begin Tracking the Learner
Exactly how can you demonstrate competencies with this technology? Here’s an example that is a common business case for our Waypoints in-line training product:
Many organizations historically use software manuals (or paper), step-by-step instructions, and e-learning video simulations to provide training for their business critical software. One organization decided it was time to modernize their business critical software training and released a new in-line training approach that will teach all employees how to use their business critical software. This is a vital training need especially when onboarding new employees. When learners use this in-line training, they perform activities in an overlay that is directly on top of the software. A variety of activities are performed including simulations, guided step-by-step processes and workflows, games, and quizzes. Activity stream data provides contextual information about the learning experience. The real assessment of knowledge happens once the learner has completed the training and they are asked to perform each software task on their own. This provides a competency based assessment which is also tracked. All in all, every one of these actions or steps that happen within accomplishing objectives, now translate to an activity stream for each learner.
- Any user activity or behavior that involves anything digital can be translated into xAPI “statements” that combine to form an “activity stream”
- The xAPI statements are sent into a database called a Learning Record Store (LRS) where they can be used to inform the organization
* In this case, the technology behind “tracking the learner” involves using Experience API (xAPI) and a Learning Record Store (LRS).
“Listen” for Competency
In our example from above, as the user is guided through the software, enough data is collected to evaluate if they’re performing as expected. The endpoint of the software training includes the “completion” and much more evaluation data about the learner. All the user’s software activities were captured until the user completes the software training. With this type of training experience the user proves their competency, which is innovative in and of itself. By reporting on all of these activities, you are able to sustain and improve the training because you also have better informed formative and summative evaluation data.
In order for this type of approach to work efficiently and effectively, the instructional design process includes determining the rules for competency based assessments. With our Waypoints product and the use-case of software training, we are able to directly map training to performance because learners are using the software and we can verify that they are performing correctly. In other areas of your digital training this may not be so easy, but still not impossible. If you are measuring performance in some other system you can understand specific training that is affecting specific performance if you are able to get better training performance data (learning activity streams). It’s important to note that information about howan objective was achieved can be more valuable than simply knowing if the objective was achieved.
Evaluate the Competencies and Take Action
What happens after you get all of your current training to report valuable activity stream data into an LRS database? After data is collected from all of the learners of the digital training then the educator, stakeholders, and managers can now evaluate and report the results. It can be valuable to provide actionable data views within today’s organization and training and evaluation data can have many audiences. Our Storepoints LRS includes the ability to directly query the database and build visualizations, construct dashboards, and distribute these dashboards as live data views or static reports.
Distribute live data dashboards or snapshot reports of data with Storepoints LRS
Let’s say the educator/trainer from the above in-line training example wants to determine a baseline of effectiveness for the training. The trainer can look at how much time and how many steps it takes an expert to complete the competency assessments and compare that to the same activity data for the total population of learners. This can be done for all the workflows but is especially valuable for those business critical activities. If it takes the expert 5 minutes and 12 steps to complete a business critical task then the total population in aggregate, certain roles, or even individuals can be easily be compared to this expert baseline. For further sustainment and improvement of the training, rules can also be created based on the activity streams to automate intervention and remediation for the learners. These are not future concepts they are available today, and it is faster and easier than you might think to put into place. Happy evaluating!