We are pleased to submit this Solution to the NIST Virtual Public Safety Test Environment Challenge. Our Solution, the Open Virtual Environment for Responder Testing (OVERT), is based on an extensible 4D simulation architecture, and the HSEEP Framework for exercise planning, design, conduct, and evaluation. It is designed to be community driven, scalable, and sustainable. OVERT integrates data logging and research capabilities, to support testing, training, exercise, and evaluation scenarios. It supports real time, time scaled, and time stepped, high and low resolution simulations, with virtual replay. During simulations, situational awareness is maintained by the use of virtual and augmented reality views, with concurrent video overwatch. The OVERT tools and technologies are designed to be easily integrated with local responder equipment and procedures, indoor location technology, and virtual sensors. Ease of implementation, and safe use by local personnel, will enhance local OVERT testing, training, and evaluation methodologies. The following sections introduce OVERT features and capabilities, which are described more fully throughout this Solution.
REALITY: The OVERT design provides for both high and low resolution rendition of scenarios, scenes, and vignettes. Simulations can be deployed on conventional computer displays, large screen and panoramic displays, augmented reality (AR) glasses in physical training spaces, and on virtual reality (VR) headgear in open spaces indoors and out. The immersive nature of the scripted scenarios can provide a high degree of realism for emergency responders and for incident command staff, whether used for training or testing.
AVAILABILITY: All of the hardware and computer components of the OVERT architecture exist and are commercially available. The game engine upon which OVERT is built is available for licensing, and the standards, software, documentation, and training to support it, needs to be completed.
VERSATILITY: OVERT has the ability to accommodate a variety of scenarios, scenes and vignettes. The architecture is designed to enable and encourage customization of scenarios with local content and details. As a community supported effort, OVERT will benefit from contributions from emergency response agencies, and OVERT users, around the country.
METRICS COLLECTION: The OVERT architecture is designed with an embedded research and data logging component. Whenever simulations are running it is collecting granular simulation log data, operational data, responder performance data, environmental sensor data, rich media data (audio, video, etc.), geospatial and position data, and test data for equipment, devices, and tactics.
REPLICABILITY: As an open architecture, OVERT is designed expressly for ease of replication. The core back-end server processes can be hosted by test labs, states, counties or local jurisdictions. To expedite installation and configuration of the OVERT architecture a set of Reference Implementations will be provided. These Reference Implementations can be used for demos and pilot programs, and can be downloaded to create turnkey local implementations.
REPEATABILITY: Once defined and configured, OVERT simulations can be started quickly. The only latency for the responders is from network bandwidth and local area network connections. Even in situations where OVERT is hosted outside a local jurisdiction, the simulation content is downloaded and cached on a local server to assure responsiveness. If necessary, simulations can quickly be stopped, reset to a prior checkpoint, and replayed seamlessly under local administrator control.
INTEROPERABILITY: OVERT is architected to support an almost unlimited range of hardware interfaces, communications, and legacy computing systems. It accomplishes this through standards, interfaces, and custom sensor interface hubs, which enable local staff to integrate devices, data feeds, imagery, video, audio, communications, and sensors into simulations.
SAFETY: The inherent benefits of simulations create a safer environment for training and testing, by reducing or eliminating hazards, and by mitigating risks. OVERT simulations have built in Operational Risk Management strategies which are part of the scenario design process. Every OVERT simulation begins with a Safety briefing as part of the scenario, and frequent safety prompts can be turned on during simulations.
ABOVE AND BEYOND: OVERT provides an open architecture solution for the future that will disrupt the current simulation and testing market space, while transforming the capabilities available to local emergency response agencies. Through community support, a national clearinghouse for simulation test scenarios, and local customization capabilities, the cost and complexity of next generation simulation will be dramatically reduced. The remainder of this Solution describes how OVERT will exceed expectations.
The OVERT system will be easy to install, set up, and configure by first responder trainers, and testing labs, at the local, state and regional levels. Its features and functionality are designed to meet the highest standards of NIMS and ICS tools, PCSR capabilities, and HSEEP exercises, but also to be customizable to meet other testing requirements. This ease of implementation is due to OVERT modularity, modes of operation, its feature sets, and its extensibility. Another key aspect off the design is its ease of integration with existing exercise management software, surveillance feeds, broadband communications, and sensor systems.
OVERT employs an open architecture to make it more easily implemented, replicated, understood, and customized. Its transparency makes it easy to understand. All of the elements which control a simulation are documented and accessible. Its openness and modularity makes it highly customizable to meet local needs. Its architectural design makes it scalable in multiple spatial and temporal dimensions. It is a 4-D simulation architecture that provides 3D rendering, in a rich geospatial context, with the addition of time scaling as the fourth dimension.
OVERT consists of components which make it especially easy to install and use. It is intended for use by emergency responder agencies and testing labs. For each of these user, there will be a Reference Implementation. Each Reference Implementation will be a full scale, live, demonstration back end system, for testing and evaluation purposes. Interested agencies will have no delays in getting hands-on access to OVERT for implementation on their own servers. Or, they can subscribe to a hosted, cloud based system, and avoid the costs and complexity of maintaining their own servers.
EXAMPLE 1: A county Sheriff’s department could try out the NIST County OVERT Reference Implementation by filling in a set of online configuration and customization forms. After completing the site customization steps, they select a simulation scenario. Following scenario selection they will customize the scenario, identify simulation managers, and establish geographic references and time parameters by filling in a few more online forms. Next, they will enter participant lists, and their organizational assignments and roles. Finally they will identify to the OVERT system the equipment configuration for computers, displays, head gear, communications interfaces, etc. The OVERT system will then contact the designated simulation managers just prior to the scheduled start, will check in other responding personnel, and will commence the simulation. Throughout the simulation, real time injects will be made as specified in the Master Scenario Event List. During the simulation, rich status and progress data is provided in real time to the local simulation managers. Following the simulation or test scenario, data logs, and many other information and analytic reports will be provided to the local simulation managers for their use in exercise debrief and evaluation. Duplicating scenario runs will be especially useful in comparative testing of equipment, tools and tactics.
EXAMPLE 2: When setting up a simulation, hardware devices can be made a permanent part of the OVERT configuration. All of the permanently installed equipment could be configured once, and thereafter would be available for any simulations run. To illustrate the OVERT customization strategies which are supported, let’s consider the use of radio communications channels, video feeds, alarms, Amber Alerts, and CAP Alerts in simulations to provide greater realism. OVERT Simulation Managers can choose from three different configuration strategies for these inputs. First, they can choose to use the existing radio communications devices issued to each responder and provide radios for the incident command (if it is part of the simulation). If so, they will need to allocate channel space for the simulation and assure that it does not interfere with live radio traffic during the simulation. They will also have to assure that the existing radios can be safely and conveniently used with the simulation headgear and equipment. Second, they could choose to use only OVERT communications channels, during the simulation. This would allocate one or more VOIP channels, dedicated to the simulation, and running on local computer hardware, headsets, and head gear. In this case, there is no conflict with local radio frequencies or need to allocate actual RF channels. Third, they could choose a hybrid approach where several real radio communications channels will be used, bridged to, and integrated with, VOIP channels used on the simulation computers, headsets, and head gear. To accomplish this bridge, a simulation computer is interfaced to a radio speaker and microphone port. These capabilities could also be employed to evaluate and test features, equipment, and other aspects of the PSCR broadband network.
Safety is a primary motivation for the OVERT design. By replacing dangerous environments in first responder equipment and personnel testing, training, and evaluation, OVERT is inherently safer in its use of virtual reality and augmented reality scenarios. Many of the hazards present in a “live” test and training environment can be eliminated by virtual and augmented reality scenes. When employing VR, the OVERT system can virtually represent hazards and dangerous conditions. When employing AR, the system can depict hazards and dangers virtually on the heads-up first responder display. When OVERT is used in Incident Command scenarios, hazards and dangers can be represented iconographically on the incident management and overwatch displays.
The OVERT system is a virtual test and training platform that will be:
Focused: Equipment under test (EuT) will be able to be run through multiple OVERT scenarios tailored to its features and capabilities. The EuT will be subjected to varying conditions and situations to determine its most optimal usage. This ability to complete consistent test scenarios will result in best practices for the employment of equipment in incident response.
Scalable: OVERT will provide highly scalable capabilities. First Responders will be able to train individually for single agency incidents, for complex single and multiple site incidents, for incident command staff training, as well as with responders from other agencies, disciplines, and jurisdictions to prepare for a coordinated incident response. OVERT will also support concurrent use by first responders, specialized personnel, and incident command staff, in multi-layer exercises.
Comprehensive: OVERT content will be supported by a national developer community. Responders will have access to a repository of pre-designed, commonplace scenario templates, which can be customized to meet local needs. Scenario, scene, and vignette scripting will be supported by conventions like the use of DHS HSEEP Master Scenario Event Lists (MSEL), and the Community Alert Protocol (CAP).
Cost-effective: As an open platform, the OVERT Reference Implementation will be able to be downloaded by any regional, state, county, or local emergency response agency. While it cannot replace some dimensions of the interactions involved in live training, it will significantly reduce costs by providing a free, flexible and extensible tool. The ease with which scenarios can be customized and run will encourage more use for training, testing, and familiarization than expensive live exercises.
Efficient: OVERT training simulations will allow a large number of responders to train repeatedly, both as individuals and in teams. Because it also supports simultaneous first responder and incident command staff training scenarios exercise realism is increased and costs are decreased.
Tailored: With OVERT, response agencies will be able to customize the open architecture platform, creating geo-specific 4-D environments that accurately depict the infrastructure and resources available in their own jurisdictions. They will be able to run these simulations in realtime, scaled time, or in time stepped modes.
Safe and Easy to Use: By removing the physical and kinetic effects of hazards from simulated exercises and drills, safety is directly improved. By incorporating Operational Risk Management (ORM) components into all simulations, proper focus on safety, hazard identification, and risk mitigation will be a part of every OVERT simulation.
Realism in simulations comes from a number of factors: the fidelity of the display and audio devices, the physics engine employed in the simulation, the scope of the simulation, the details and specificity, and from the responsiveness and interactions in the simulation. Research has shown that both high and low resolution simulations can have beneficial outcomes if they include a high level of realism. Well scripted, low resolution simulations that are highly responsive and interactive can seem more real than some high resolution simulations with poor interactivity, lack of detail, narrow scope, or sluggish performance. OVERT addresses these issues by providing a virtual sliding scale for reality, and potentially different levels of reality for participants.
OVERT is designed for simulation and testing in all of the “Levels of Response”. Each Level of Response involves different competencies. Scenario and vignette designs can be engineered to facilitate Awareness Level Response, Operator Level Response, and Incident Command Level of Response. See the illustration of the Simulation Hierarchy in the attached document. One simulation can address a single Response Level, or it can simultaneously render multiple Levels of Response.
One of the important features of OVERT is its ability to scale reality through the use of various virtual display technologies. So, for example, in a complex multi-scene incident the responders at Scene A may be using VR headgear to operate within a “virtual world”, while responders at Scene B ma be using AR headgear to operate in a “mixed reality world”, and incident command staff are using remote communications and surveillance video (real and virtual) to manage the incident. In such a multi-layered simulation, reality would scaled in three different ways. The Augmented Reality component of OVERT is especially valuable in adding hazards and technical challenges, and to layer them over real scenes.
OVERT is not a ‘one size fits all’ system. Its design and fundamental architecture supports multiple simulation scopes, concurrent alternate realities, and powerful time scaling for efficient use of resources. Time scaling provides for simulations to run in real time, altered time which can be sped up or slowed down, and stepped time which can create discontinuities in time by jumping forward and backward. Typically exercises and tests are conducted in real time. This replicates reality, but often creates inefficiencies and delays as one team is forced to wait for other events to happen. Scaling time in these cases permits the simulation to “fast forward” or “rewind and replay” elements of the simulation. This time scaling can affect the entire simulation, or can affect different scenes differently. Sometimes there are just 5 minutes during a simulation hour when key decisions need to be made or critical actions taken. Time can be morphed for responders to focus more on those 5 minutes, and possibly enable them to repeat them several times, while the rest of the simulation continues in real time. Time stepping enables responders to practice skills, or test equipment and tactics, in an economical way by “fast forwarding” over less important aspects of the exercise like staging and wait times. Using these time altering strategies can mimic the way the mind works during real emergencies, when responders report that time seems to slow down and events take place in slow motion.
OVERT will be implemented using a proven simulation engine capable of immersive Virtual Reality, support for Augmented Reality hardware, multiple virtual views, and a powerful Application Programming Interface (API) to extend the feature set, and to add the 4-D time scaling features. The Unreal Engine and the Unity Engine have both been used in public safety training simulations and are the leading candidates for OVERT. Our preference, based on the richness and extensibility of its API, is the Unity engine. The support for interoperability standards by the Unity Game Engine includes HLA, DIS, and RMI. These features open up possibilities for reuse of DOD simulation content and infrastructure.
The Unity engine has scalable artificial intelligence driven non-player characters, built in collaboration tools, a context-sensitive user interface, a physics controllable kinematic skeletal animation system, DirectX 11 support, fire, particle, and smoke propagation system, weapons equipment and tools models, and character damage and injury models. The Unity engine also has administrative and management tools and APIs, an interactive dialog system, inventory tools, vehicle support, customizable player classes, multi-player networking, voice over IP (VOIP) communication, and embedded video streams. The Unity Game Engine real time scripting engine API will enable OVERT to manage and control simulations, collect data, and control time and other important features.
To understand the full value of OVERT it is important to know its various modes of operation, what in it is real, and what is virtual, who the actors are, and the options for its scene staging. These modes of operation reveal the many flexible ways OVERT can be employed.
Let’s begin by looking at scenarios involving testing and training with Incident Command Staff. For simple exercises this may include an on-site Incident Command. For more complex scenarios it might include a more complex Unified Command or Area Command. To replicate a typical Incident Command Post (ICP), the OVERT ICP is staffed and has access to computers, communications channels, and video feeds. These incident command tools can be real, virtual, or a combination of both. Depending on the scenario it may include real or virtual first responders at a real or virtual incident site. As you can see, even in this simple case a great amount of flexibility and reusability can contribute to the economy of an OVERT simulation exercise.
Next, let’s look at the first of three different First Responder modes of operation. In this mode, the responders arrive at a virtual scene in a physical set, with a CAVE or hemispheric projection screen to create a virtual reality simulation of the scene. The responders interact with the actors in the screes surrounding them and communicate with a real or virtual Incident Command Staff via communications devices (radios, phones, etc). There are limits to the use of this type of physical set, but the responders are unencumbered by any head gear or cables. A variation of this mode is the computer based simulation where a 3D virtual set is presented on a computer screen, with simulated communications to a virtual command post. There are currently many third party, computer based, first responder training scenarios.
A second mode of operation for First Responders is an Augmented Reality mode. In this mode the responder uses actual physical equipment, in a reusable, specially designed, indoor or outdoor set. The set contains basic elements of the incident scene, and may contain live victims or other actors. The responders wear AR head gear through which they can view the scene with augmented reality annotations, hazards, and even virtual victims and other actors. Responders communicate using VOIP capabilities of their head gear, or real radios or phones, and the responders interact with one another naturally. This mode of scenario is good for equipment, tactics, and skill testing, and makes good use of augmented reality features to keep responders safe from hazards by simulating them. Augmented Reality can add fidelity to a low res simulation.
The third mode of operation for First Responders is the Virtual Reality mode. In this mode the responder wears VR head gear and the scenario exists exclusively inside the 3D simulation. This mode of operation is typically executed inside a large open building or warehouse or outdoors in an open area. These settings provide ample space for the responder to move around, for their movement to be tracked, and for them not to bump into anything and remain safe. Responders communicate using VOIP capabilities of their head gear with incident command staff who may be real or virtual. Using this mode of simulation, virtual tools and equipment can be tested. High fidelity in rendering of the 3D virtual set is necessary and the responders are represented by avatars.
To manage simulation activity in OVERT a user/administrator defines all of the real world and virtual components in an Incident Framework Description, and then adds a hierarchy of Scenarios, Scenes, and Vignettes to that Description. The scope of the Incident Framework can range from a simple single site scenario, to a more complex single site scenario, to a simple multi-site scenario, to a complex multi-site scenario, to multiple concurrent multi-site scenarios. Regardless of the scope, each incident modeled has two elements: the first responders and the command and control center. OVERT needs to be able to simulate command center actions, and it must be able to integrate real people into the command center roles. Although the focus of most training and testing is on the first responders, OVERT must always provide structure and connectivity through the command center entity and its resources and communications channels.
Another dimension to look at in OVERT is the type of exercise, according to the DHS HSEEP taxonomy. OVERT will support both Discussion Based Exercises (DBE), and Operations Based Exercises (OBE). In DBE, OVERT supports seminars, workshops, tabletop exercises, and interactive games. In this domain the AR, VR, and 3D graphics are not as important. In the OBEs, Drills, Functional Exercises, and Full Scale Exercises are supported. As in all of these HSEEP types, OVERT will support enhanced Master Scenario Event Lists (MSEL) as the organizing metaphor for simulations.
In order to answer how OVERT will enable the testing of multiple tools, techniques, technologies, and interfaces, it is necessary to review the modes of operation and to discuss the testing in context. The first mode of operation utilizes computer displays and projected large screen displays providing an immersive environment for testing. The second mode of operation is used in a physical setting designed for responder testing with the use of augmented reality headgear to enhance the simulation and virtually represent the hazards and other effects. The third mode of operation is used in open space indoor and outdoor environments for responder testing with the use of 3D virtual reality headgear to immerse the responder into a virtual environment containing all elements of the scenario.
As we understand this Challenge, the system architecture should support robust testing of “entities under test” under all OVERT modes of operation. In the implementation of OVERT, there will be some modes of operation that are better suited for testing some entities, rather than others. The taxonomy of entities under test should include: physical equipment, devices, and tools under test (PuT), augmented reality interfaces under test (AuT), virtual reality interfaces for RuT under test, and virtual equipment, tools, and interfaces under test (VuT). Another class of virtual test entities includes: device and sensor concepts under test (CuT), tactics and procedures under test (TuT), and competencies and skills under test (SuT).
Mode 1 an 2 of OVERT operation are better suited to testing pf physical equipment, sensors, devices, and tools (PuT). Modes 2 and 3 are better suited to testing of virtual reality interfaces for RuT under test, and virtual equipment, tools, and interfaces under test (VuT). All three OVERT modes of operation are well suited to testing of device and sensor concepts under test (CuT), tactics and procedures under test (TuT), and competencies and skills under test (SuT).
There are five methods employed in OVERT to accomplish testing: observe and monitor the use of the physical entity under test with overwatch video, interface the physical entity under test to the system, interface a physical controller for the entity under test to the system, representing the physical entity under test with a virtual device interface, and duplicating physical entity capabilities in a virtual entity or with augmented reality.
When thinking of OVERT, it is important to understand that in any one simulation instance there are many more tools and capabilities available than those included in a scenario. The test or training requirements will determine which capabilities and features are turned on, and which data is to be automatically collected. The following test use cases illustrate how OVERT can be used to test responder equipment interfaces, data collection tactics, and simulation analytics.
TEST USE CASE 1:
OVERT will support many types of display environment including touch displays, heads-up AR displays, VR displays, panoramic large screen displays. New haptic display types can be tested in live simulations. Various UI capabilities will be supported including audio and visual cues, voice commands, intelligent responder assistants, haptic and gesture controls, key pads, touch controllers, wrist keyboards, wearable computers, and differing interface layouts, features, and designs. User interface elements can be tested in live simulations.
TEST USE CASE 2:
OVERT will support man types of I/O devices, sensors, data collection and communications devices, including cameras, thermal imagers, video capture, radios, tablets, smartphones, mobile displays, and tagging and input devices. In addition a whole class of sensors will be supported and new ones will be able to be added. Sensors ranging from radiation and temperature detectors, to gas sniffers and geo-location devices, and biofeedback sensors. Virtual versions of these I/O devices and sensors may also be included in scenarios for use and testing in place of expensive physical devices.
TEST USE CASE 3:
An important OVERT feature is its deep research and data logging capability, which is built into the architecture. During a simulation a vast amount of data will be collected. The simulation framework will define the requirements for which data is analyzed and reported. All the data is archived for research purposes, system improvements, and future use. The classes of simulation information collected include granular performance data, evaluation metrics, event and action logging, geospatial data on positions of all entities, audio and video data, event data, imagery and sensor data. Much of this comprehensive data can also be made available to responders and incident command staff, through their user interface during simulations, to provide better situational awareness, to facilitate blue force tracking, risk management, and hazard avoidance.
The benefits of OVERT are based on shared collaborative development and community ownership of the technology. NIST and large Public Safety organizations have resources, personnel, and budgets to develop standards and open architecture simulation components. By bringing NIST together with states, large cities, counties, test labs, industry, and academia, into an OVERT Consortium, these agencies can use and contribute their resources and creativity to the evolution of the architecture. By combining those capabilities with an open standards based architecture, and community development and sharing, has the potential to bring high quality simulation technology for emergency responder testing and training to all size organizations across the country. Thousands of small businesses and local jurisdictions will be able to benefit from this transformative model.
The national OVERT Consortium will establish new standards, recommend hardware, equipment,, software, and testing procedures that are OVERT compatible, publish generic customizable simulation content, conduct training and conferences, publish testing best practices, and provide a governance model for the evolution of the OVERT architecture. It will also support interoperability with existing standards like the First Responder National Standard Curriculum from DOT, NHTSA, and DHHS. It will also provide an impetus to assure that OVERT has interfaces to recognized software like the NIST Exercise Control System (ECS), the broadband PSCR initiative, and that it will support integration with, or emulation of, existing DHS systems like the Community Alert Protocol (CAP).
The following line items represent the one time investment in building infrastructure and doing the foundation building work, that will pay off as testing organizations and jurisdictions implement OVERT. Costs are not specified as they are highly dependent on NIST decision making. Investments in national OVERT infrastructure, and in establishing the OVERT Consortium, should disrupt the Simulation marketplace and dramatically reduce the costs of setting up OVERT tools and technologies for local jurisdictions. That will guarantee much wider dissemination of the benefits of OVERT technology, sooner.
National OVERT Infrastructure and Program Support
-- National OVERT Website
-- Hosted OVERT Reference Implementations
-- OVERT Clearinghouse (documentation, training, simulations, best practices, content assets)
-- OVERT Architecture and Integration Activities
-- OVERT Research Development and Training
-- Game Engine (Enterprise License)
The following shopping list illustrates the components required to install and operate OVERT in jurisdictions across the country. The quantities will depend on the size of the Emergency Response Agency. All components shown are commercial over- the-shelf (COTS), and undiscounted unit cost estimates are provided. These lists do not include regular issue responder equipment or any other training consumables. Nor do they include staffing or operating costs.
State, County, or Local OVERT Installation at Emergency Response Agency
-- Training facilities or Open multi-use spaces…………. varies
-- Servers with OVERT Installed ………………………… 1600
-- Game Engine License …………………………………. 400 annual
-- Large screen display or panoramic projector ……….. optional
-- Workstations ……………………………………………. 800
-- Laptops ………………………………………………….. 500
-- Tablets …………………………………………………… 400
-- AR Glasses ……………………………………………… 600
-- VR Headgear ……………………………………………. 600
-- Locatioin sensors ……………………………………….. 400/100
-- Biometric sensors ………………………………………. 350
In considering these costs, it is important to explain our decision to incorporate capabilities for both training and testing simulations into OVERT. The primary reason for linking these requirements is the substantial cost savings the combination yields. Another reason to combine test and training is that significant information can be collected during training, which will yield baselines for specific future test scenarios. There is also synergy from supporting, training and testing in the same system architecture. The simulations for testing purposes are necessarily more replicable and realistic, and the training simulations will be able to benefit from that realism.
In the current Simulation marketplace, local jurisdictions need to pay for expensive tools and closed proprietary authoring systems, need to find and train simulation designers or contract that out, get locked into dead end hardware solutions, and sometimes are not able to create high quality simulations at all. The OVERT architecture, the OVERT Consortium, and the OVERT clearinghouse will all work together to disrupt that market place and reduce the costs of high quality simulations, while increasing their number, and improving their quality. The idea that NIST, and large emergency response agencies, will work together to develop a wide selection of free, customizable, high quality simulations, and post them to the OVERT clearinghouse repository, will transform the Simulation market. For a modest one time investment in hardware, local emergency responders will gain access to world class simulations they can customize to meet their local needs. The time consuming effort of defining and configuring rich 4-D simulations only will need to be done once, and then it can be shared to benefit everyone.
OVERT employs a hierarchy of Scenarios, Scenes, and Vignettes. Scenarios are complete simulations with specific goals, content and outcomes. They may contain one or more scenes. Scenes are site settings or locations within a scenario. In simple scenarios there may be only one scene. Man scenarios have multiple scenes. Vignettes are interactions which take place at a scene. For example at the scene of an automobile accident one officer may be directing and diverting traffic, another may be interviewing witnesses, and an EMT may be helping a victim. In this scenario scene there are three vignettes. Each vignette has specific actors, tasks, conditions, standards of performance and outcomes. In testing and training we measure the outcomes of the vignettes. The following sections describe a representative set of first responder situations which can be simulated for testing or training purposes using OVERT.
Firefighters - Public building. This scenario is best suited to an augmented reality simulation using a physical set with hazards and victims represented in the AR glasses.
Police - Traffic stop. This scenario is best suited to a virtual reality simulation using an open space with vehicles and streets depicted and experienced through the VR headgear. All aspects of the traffic stop protocol should be implementable using the VR simulation.
Police - Crime scene walk through. This scenario is best suited to a virtual reality simulation using an open space, or an augmented reality simulation with victim, evidence, and other investigators depicted and experienced through the VR headgear or AR glasses. All aspects of the crime scene investigation protocol should be implementable using these simulations. A virtual measurement tool, sample collection capabilities, virtual black light, a virtual camera, and virtual Lidar 3D image capture tool could be used to recreate the undisturbed scene later. Simulation replays could be used to capture details missed or evidence contaminated.
EMS - Accident Response - This scenario is best suited to a virtual reality simulation using an open space with vehicles, highway and traffic depicted and experienced through the VR headgear. All aspects of the EMS response protocol should be implementable using the VR simulation. Virtual video conferencing, virtual cameras, virtual communications devices, and virtual medical support devices may be helpful in this scenario.
Search & Rescue - Unmanned aerial vehicle (UAV) - This scenario is best suited to a virtual reality simulation using an open space, or an augmented reality simulation with the mountain wildfire, other personnel and equipment depicted and experienced through the VR headgear or AR glasses. All aspects of this SAR UAV protocol should be implementable using these simulations. A virtual UASV controller, virtual or real UAV, control communications and virtual camera and video feeds could be used. Simulation replays could be used to capture details or evidence missed.
Explosives and Hazmat - Unmanned ground vehicle (UGV) (aka ‘bomb robot). - This scenario is perfectly suited to an augmented reality simulation using a physical set with the UGV, obstacles and hazards represented in the AR glasses. Responder will control the virtual UGV in the environment with a real controller interfaced to the simulation, taking sensor readings and virtual video.
Cyber First Responder - Crime Scene - This scenario is best suited to a virtual reality simulation using an open space, or an augmented reality simulation with computers, video evidence, and other devices depicted and experienced through the VR headgear or AR glasses. All aspects of the Cyber crime scene investigation protocol to survey and secure electronic evidence should be implementable using these simulations.
OVERT, the proposed Solution Architecture, has embedded research infrastructure dedicated to data collection and analysis. The default mode of the system is to collect granular data simultaneously from users and system components. Selection of specific data collections, during the creation of Scenario Frameworks, will make that data available to the data analytics and reporting systems for use in debriefings, test and exercise evaluations, equipment test reports, responder competency profiles, and simulation design improvements. In this section we will identify the specific classes of data collected for analysis. Collection of these data classes will be included with the initial implementation of the architecture. Data will be collected from first use and testing. Much of the detailed data analytics will be completed later, for use in system tuning and other applications.
This section will also explain what types of data will routinely be collected and will be available for research analysis. It is hoped that OVERT University partners will be interested in harvesting this rich data resource. The classes of data collected will be simulation log data, operational data, responder performance data, environmental sensor data, rich media data (audio, video, etc.), geospatial and position data, and test data for equipment, devices, and tactics. Associated with competency maps for every responder type and and incident staff position, there will be standards of knowledge, skills, attitudes, and performances, which measure mastery. Each time responders participate in a simulation scenario their knowledge, skills, attitudes, and performances are assessed and recorded to update their individual competency profiles. Likewise, equipment and other entities under test will have a requirements map, and every time that entity is involved in testing its entity test profile will be automatically updated. Low assessments of responders will result in updated individual learning plans for additional training. Low assessment for test entities will result in entity modification requests and/or training plan updates.
Examples of simulation log data collected are: Engaging with walls, doors, furniture etc., things in real life that must be handled in virtual space, measuring mistakes made with the technology interface in Virtual space, precision and granularity of gestures and voice commands. Assets injected into the simulation scenario.
Examples of operational data collected are: Efficiency of interactive conversations and tasks between first responders and command center. Speed of command center to access information from scene, make decisions, and send directions back to the first responders. What level of information or data is actually required for optimal performance? Address data overload vs. not enough to make a good decision? Benchmarking success for tasks. Misunderstanding rate and rate of requests resulting in incorrect action. Reattempt rate (measure of user impatience). Speed from communication to resulting action. Enabling operations vs. distracting from operations
Examples of responder performance data collected are: How quickly can you get to a person in a fire and extricate them from scene? Improve safety of officer with delivering records and info. in a quick way—confirmation that an item has been stolen, or a fugitive has been found? Minute-by-minute EMS measurements, providing best care with prioritization. Balance care of multiple people on a scene. Optimizing care beyond visual assessment, using technology that tells more detailed medical information? Victim identification, medical record receipt and display? Accuracy of completing a task? Set-up time at an emergency scene? Taking pictures of a scene, saving it, and sharing to appropriate staff while protecting private data as required for specific departments?
Examples of environmental sensor data collected are: CO, CO2, NO3 readings. Virtual sensor data provided to responders in simulation. (temperature, pressure, dew point).
Examples of rich media assets collected are: Upload and/or sharing of visual elements—photos, video, etc. Captured and logged data from a crime scene or accident scenario. Overwatch video of the simulation set with clock overlay. Body cam video. Scenario street cam video. Communications audio for all channels (time stamped). Infrared images of wreckage. Virtual imagery. Simulation video from a participant viewpoint. Drone footage.
Examples of geospatial and position data collected are: accuracy and precision of location based technologies. Street maps and charts. Tide tables. Moon phases.
Examples of other entity test data collected are: measure multiple technologies on one task—such as haptic, visual, auditory, to determine which is most effective to complete the task. Another measure is the accuracy of the information that is collected.
