N251-023 TITLE: Live, Virtual, Constructive (LVC) Afloat: Automated Scenario Generation

OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Human-Machine Interfaces;Trusted AI and Autonomy

The technology within this topic is restricted under the International Traffic in Arms Regulation (ITAR), 22 CFR Parts 120-130, which controls the export and import of defense-related material and services, including export of sensitive technical data, or the Export Administration Regulation (EAR), 15 CFR Parts 730-774, which controls dual use items. Offerors must disclose any proposed use of foreign nationals (FNs), their country(ies) of origin, the type of visa or work permit possessed, and the statement of work (SOW) tasks intended for accomplishment by the FN(s) in accordance with the Announcement. Offerors are advised foreign nationals proposed to perform on this topic may be restricted due to the technical data under US Export Control Laws.

OBJECTIVE: Develop the capability to automate the creation of training scenarios for aviation platforms underway using the Next Generation Threat System (NGTS). This capability should include a framework and accompanying architecture in which specific performance parameters can be varied based on Machine Learning (ML) algorithms and Artificial Intelligence (AI).

DESCRIPTION: As the Navy prepares the carrier airwing of the future (AWOTF) for the high-end fight, the training paradigm will shift to almost exclusively Live, Virtual, Constructive (LVC) environments due to expanded range capabilities of the peer threat competitors and Operational Security (OPSEC) considerations. As a result, warfighters will be able to train as they fight with higher fidelity scenarios that more accurately represent red kill chains. This high-fidelity, data-rich environment provides unique opportunities for instructional strategies to better support end-to-end training and improve readiness. Specifically, LVC environments increase the amount of—and access to—data that can support improved scenario generation, performance assessment, and debrief when utilized appropriately. However, LVC training is not without its challenges. These challenges include resource requirements to develop these high-fidelity scenarios as they can be cumbersome and labor intensive. Moreover, scenarios that do not contain significant variations may lose utility very quickly as operators can begin to anticipate scenario outcomes after a few exposures. Consequently, a need exists for rapid generation of real-time, adaptive, high-fidelity scenarios.

Additional challenges lie in the assessment of performance. The carrier airwing of the future will rely on integrated tactics that require a level of coordination and information exchange across platforms that have not been required in past tactics. The complexity of coordination associated with integrated tactics necessitates a significant amount of voice communications across the different platforms to provide Situation Awareness (SA) and elicit decision-making. While communication is critical to cross platform coordination and overall tactical execution, it remains one of the most challenging training objectives to meet during Air Defense events.

As such, the present effort seeks to alleviate identified challenges with scenario generation and performance assessment through the investigation of generative AI (e.g., DALL-E and ChatGPT) or other forms of AI to support scenario generation and communications assessment. This SBIR effort shall focus on utilizing AI to learn from pilot-in-the-loop red threat behavior to rapidly generate constructive threat presentations that adapt to trainee behavior in a tactically feasible manner. Additionally, AI shall be applied to further the state-of-the-science in communications analysis [Ref 5]. Specifically, AI shall support analysis of blue recorded communications and provide an initial assessment in terms of accuracy of the words said (relative to ground truth) and speed at which they are said. This will include digesting communication recordings, assessing quality of communications-based accuracy and speed, then providing these results via automated debrief.

These capabilities will improve the quality of training and readiness via end-to-end training enhancements. First, high-fidelity Air Defense scenarios that can be rapidly generated and are adaptive will yield greater training utility and provide cost avoidance associated with scenario development manpower and human-in-the-loop (HITL) threat support manpower. Next, development of a communications analysis and debrief capability will improve SA, and decision making will benefit the Fleet by decreasing instructor workload, reducing human error and manpower time requirements, and automatically provide instructors with information on communication protocol adherence and timeliness to improve SA and increase debriefing capabilities.

This effort will specifically look at Air Defense training scenarios within LVC environments to increase speed at which high-fidelity, adaptive scenarios can be generated and assessed to enhance operator performance. This capability will be developed with the intention of a transition path to the NGTS.

Work produced in Phase II may become classified. Note: The prospective contractor(s) must be U.S. owned and operated with no foreign influence as defined by 32 U.S.C. § 2004.20 et seq., National Industrial Security Program Executive Agent and Operating Manual, unless acceptable mitigating procedures can and have been implemented and approved by the Defense Counterintelligence and Security Agency (DCSA) formerly Defense Security Service (DSS). The selected contractor must be able to acquire and maintain a secret level facility and Personnel Security Clearances. This will allow contractor personnel to perform on advanced phases of this project as set forth by DCSA and NAVAIR in order to gain access to classified information pertaining to the national defense of the United States and its allies; this will be an inherent requirement. The selected company will be required to safeguard classified material during the advanced phases of this contract IAW the National Industrial Security Program Operating Manual (NISPOM), which can be found at Title 32, Part 2004.20 of the Code of Federal Regulations.

PHASE I: Design and develop a development plan for a proof-of-concept solution to rapidly automate the creation of training scenarios for aviation platforms at sea. This will include identifying the most appropriate methodology for developing the scenario generation framework and architecture, enhancing red threat presentation, and developing a graphical user interface (GUI) for instructors that align with current NGTS GUI. An unclassified sample dataset will be provided to help support this investigation. The Phase I development plan will be used to demonstrate feasibility of application into the larger, integrated training system. The plan shall detail integration into NGTS to allow for transition into an operational LVC environment. Additionally, the plan shall include a Subject Matter Expert (SME) evaluation of capabilities and how to conduct an Analysis of Alternatives to identify best practice method moving forward for training delivery. The Phase I effort will include prototype plans to be developed under Phase II.

PHASE II: Research, develop, design, and deliver a proof-of-concept scenario generation capability and intelligent mechanism for modifying threat models based on evolving threat presentations and test in NGTS through execution of the integration plan developed in Phase I. During Phase II, the sample data provided will be more tactically and operationally relevant and classified at the SECRET level. Awardees can expect the scenarios to be more tactically complex. Design and tool development shall include tactically appropriate presentation of evolving red threats utilizing the NGTS behavior structure, usability documentation, and technology evaluation. Demonstration of the tool, along with documentation of usability of the training software is critical. Risk Management Framework guidelines should be considered and adhered to during the development to support information assurance compliance.

Work in Phase II may become classified. Please see note in the Description section.

PHASE III DUAL USE APPLICATIONS: Introduce additional data from NGTS, as well as other live and virtual entities within the scenario. Scenario generation shall be enhanced to include external (live and/or virtual) entities. Integration testing and demonstration of capabilities will be conducted in a distributed simulation via Distributed Interactive Simulation (DIS) protocol at the SECRET level. Software shall be integrated with NGTS to facilitate transition into operational LVC environment. Documentation and any supporting materials shall be delivered to NGTS team for maintenance and future enhancements.

The scenario generation tool can be leveraged in the private sector as an aviation training aid in environments with limited network access. Tailorable aviation scenarios can be used for commercial or private pilot training devices to expose trainees to a wider variety of simulated high-risk events.

REFERENCES:

1. Wang, K.; Gou, C.; Duan, Y.; Lin, Y.; Zheng, X. and Wang, F.-Y. "Generative adversarial networks: Introduction and outlook." IEEE/CAA Journal of Automatica Sinica, 4(4), 2017, pp. 588–598. https://ieeexplore.ieee.org/iel7/6570654/8039012/08039016.pdf

2. Yardley, R. J.; Thie, H. J.; Paul, C.; Sollinger, J. M. and Rhee, A. "An examination of options to reduce underway training days through the use of simulation." RAND National Defense Research Institute, 2008, pp. 1-140. https://apps.dtic.mil/sti/citations/ADA486311

3. Park, K.; Mott, B. W.; Min, W.; Boyer, K. E.; Wiebe, E. N. and Lester, J. C. "Generating educational game levels with multistep deep convolutional generative adversarial networks." 2019 IEEE Conference on Games (CoG), August 2019, pp. 1-8. https://doi.org/10.1109/CIG.2019.8848085

4. Martin, G. A. "Automatic scenario generation using procedural modeling techniques." Electronic Theses and Dissertations, 2152, 2012, pp. 1-135. https://stars.library.ucf.edu/etd/2152

5. Chatham, R. and Braddock, J. "Defense Science Board Task Force on Training for Future Conflicts." Washington, DC: Office of the Undersecretary of Defense for Acquisition, 2003, pp. 1-110. https://prhome.defense.gov/Portals/52/Documents/RFM/Readiness/docs/future_conflict.pdf

6. "Tentative Manual for Expeditionary Advanced Based Operations." Headquarters, United States Marine Corps, Washington DC, February 2021. https://mca-marines.org/wp-content/uploads/TM-EABO-First-Edition-1.pdf

7. "National Industrial Security Program Executive Agent and Operating Manual (NISP), 32 U.S.C. § 2004.20 et seq. (1993)." https://www.ecfr.gov/current/title-32/subtitle-B/chapter-XX/part-2004

KEYWORDS: Scenario Generation; Live, Virtual, Constructive; Training Underway; Artificial Intelligence (AI); Tailorable Scenarios; Next Generation Threat Systems (NGTS); Adaptive Threat Models

TPOC 1: Jennifer Pagan
(407) 380-8130
Email: [email protected]

TPOC 2: Heather Priest
(407) 380-472
Email: [email protected]

---