N221-069 TITLE: DIGITAL ENGINEERING - Digital Twins to Enable Training (DTET)

OUSD (R&E) MODERNIZATION PRIORITY: General Warfighting Requirements (GWR)

TECHNOLOGY AREA(S): Human Systems;Information Systems

OBJECTIVE: Develop an enterprise training solution that integrates digital twin models [Ref 1] and their related data with immersive training content capabilities and adaptive training algorithms to accelerate the acquisition of knowledge and increase learning gains with a focus on maintenance tasks.

DESCRIPTION: Current maintenance training systems lack several modern training capabilities: (1) easily created and modified immersive training content; (2) algorithms and technologies that enable adaptive and tailored training; and (3) an enterprise focus that allows for automated content creation across many domains, whether it be immersive content, tailored lesson plans, or adaptive tutoring through a curriculum. The current state of the art of digital twin technologies in development to design new systems could also be used for training and education. However, new capabilities are required to link the underlying authoritative models with immersive content creation pipelines that can leverage poor-quality source data in an automated fashion requiring minimal personnel interaction. Additionally, new training and education systems must be developed to allow for adaptive and tailored training [Ref 2] that can be applied to a variety of maintenance domains, and do not require specialized personnel to develop training content and curricula. A convergence of key enablers exists to pivot towards an immersive and tailored training approach by exploiting the availability of computer vision, advances in machine learning, and science of learning. Proposals should leverage emerging commercial technologies while addressing the technical challenges associated with supporting distributed military environments and training at an enterprise scale.

The overarching goal of this effort is to connect authoritative digital twin models, developed as part of an overarching Digital Engineering approach to streamline new platform development, with a generalized and domain-agnostic persistent training platform for automated content creation and tailored learning guidance at an enterprise scale connected to an eLearning system (such as Moodle). The immersive content creation pipeline may need to rely on a suite of technologies, such as object scanning, photogrammetry, and computer vision to create content usable in current and future classrooms [Ref 3]. These technologies should be able to be integrated with and leverage models from existing Marine Corps digital twin efforts. Responded are expected to have existing content for which to use for the topic. Authoring, content development, and management of the tailored training system should leverage machine learning and other adaptive algorithms.

The end state of this program is a capability to leverage already-validated digital twin models as part of a broader maintenance training tool that tailors instruction to each individual student. This program should automate source material ingest and immersive content creation reducing the time to create training curricula and increase learning gains (e.g., test scores) by creating opportunities to interact with immersive content and be guided through curricula by macro- and micro-adaptive tailored training algorithms. Human Subjects testing may be needed in Phase II to assess content creation efficiency increases and training effectiveness outcomes. The anticipated skill sets necessary to support this topic are: maintenance subject matter experts, computer scientists, software engineers, instructional designers, data scientists, and human factors psychologists.

PHASE I: Develop early mockups and prototypes for software, the associated workflow and requirements for supporting an enterprise capability for source content ingest, automated immersive content creation, and adaptive learning within a Marine Corps eLearning ecosystem (e.g., Moodle). Source content could vary from static images to 3D scans of physical objects to CAD models.

Produce the following deliverables: (1) requirements for the system components including leveraging and integrating with existing Digital Twin models; (2) methods to efficiently ingest poor-quality, limited source data and automate immersive content creation; (3) learning sciences approaches for delivery of content; and (4) overview of the system and plans for Phase II, which should include key component technological milestones and plans for at least one operational test and evaluation, to include user testing.

If exercised, the Phase I Option should also include the processing and submission of all required human subjects use protocols as needed for Phase II training effectiveness evaluations. Due to the long review times involved, human subject research is strongly discouraged during Phase I. Phase II plans should include key component technological milestones and plans for at least one operational test and evaluation, to include user testing.

PHASE II: Develop a prototype system and conduct a hands-on demonstration with Marines (coordination aided by ONR) in a designated field of maintenance (e.g., ground vehicles, radio communications, etc.). Conduct a usability assessment and perform a training effectiveness evaluation. Specifically, develop an early-stage prototype focused on no more than two task domains to support the source ingest and content creation pipeline and adaptive training technologies. Construct a survey to provide feedback from maintenance instructors and students (assistance in determining relevant population and coordinating for demonstration/field test by ONR). Collect impressions of usability, develop objective metrics of time and effort to create immersive content, and measure learning gains (including quantity and quality of acquired knowledge). Perform all appropriate engineering tests and reviews, including a critical design review to finalize the system design. Once system design has been finalized, conduct a usability test of the immersive content creation system and training effectiveness evaluation with a relevant Marine Corps population.

Produce the following deliverables: (1) a working prototype of the system that is able to interact with existing system specifications; (2) evaluation of system usability and efficiency to ingest source data and create immersive training content; and (3) a training effectiveness evaluation of system capabilities to provide demonstrable improvement to the instructor population (Human Subjects protocol needs to be approved in Phase I Option if needed for this evaluation). Institutional Review Board approval for human subjects research can take 6-12 months, and this must be taken into account if human subjects research will be part of the proposed work.

PHASE III DUAL USE APPLICATIONS: Support the Marine Corps in transitioning the technology for Marine Corps use. Develop the software for evaluation to determine its effectiveness in either a formal Marine Corps school setting or other training setting. As appropriate, focus on broadening capabilities and commercialization plans.

Development of affordable, scalable, non-proprietary technologies are needed in order to integrate immersive content creation and accelerated learning concepts across the DoD. The commercial sector is developing some of these technologies, but they often do not have critical issues regarding non-existent, limited, or low-quality source data, nor do they often address encryption and classification. This technology will have broad application in the commercial sector, such as in manufacturing and industrial equipment maintenance.

REFERENCES:

1. Jones, D.; Snider, C.; Nassehi, A.; Yon, J. and Hicks, B. "Characterising the Digital Twin: A systematic literature review." CIRP Journal of Manufacturing Science and Technology, May 2020, Vol 29, pp. 36-52. https://www.sciencedirect.com/science/article/pii/S1755581720300110.
2. Durlach, P.J. and Ray, J. M. "Designing adaptive instructional environments: Insights from empirical evidence." (Technical Report 1297). Arlington, VA: U.S. Army Research Institute for the Behavioral and Social Sciences, 2011. https://apps.dtic.mil/dtic/tr/fulltext/u2/a552677.pdf.
3. Feiner, Steven and Henderson, Steven. "Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair." IEEE Transactions on Visualization and Computer Graphics, Volume 17, Issue 10 (October 2011), pp. 1355-1368. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.182.613&rep=rep1&type=pdf.

KEYWORDS: Digital Twin; Adaptive Training; Content Creation; Maintenance; 3D Models