Support for Training Effectiveness Assessment and Data Interoperability

Support for Training Effectiveness Assessment and Data Interoperability

The U.S. military faces increasing pressure to train and maintain readiness with fewer resources. New training technologies offer the promise of lower-cost, effective training, but assessing the effectiveness of those technologies is expensive. To reduce this cost, technologies are needed to automate performance assessment.

We present a research project investigating the design of a system to automatically assess the learner performance during Army rifle marksmanship training. Recent efforts from the Army Research Laboratory (ARL) on interoperable performance assessment (IPA) for individuals and teams have made advances on defining and persisting human performance data. The effort has concentrated on leveraging the work on the Experience API (xAPI) of the Advanced Distributed Learning (ADL) Initiative in an effort to produce human performance data that has intersystem data value.

The intent of the effort is to enable the collection of data across the training continuum that 1) enables a historical view of proficiency, 2) demonstrates a live view of performance, 3) enables macro and micro adaptation, and 4) collects data for trends analysis for efficiency and effectiveness studies. 

Download Now

In this paper, we present the first phase of work in this effort which included a user needs analysis of trainers, training developers, resource managers, and researchers to determine the ways in which they would need to access and the tools they would need to use marksmanship performance data. From this, we developed a library of measures and an xAPI registry to encode this library of measures. This registry also acts as a stand-alone reference for training simulator vendors to easily create compatible, marksmanship-specific, xAPI measures. Finally, we developed an architecture for IPA data capture and usage that maps key use cases for data capture, identity management, storage, secure access, and data analysis. The use cases were employed to create the system architecture designs.