A reconfigurable vehicle frame is recommended. This will allow for training in any kind of personal vehicle (car, SUV or small pickup truck). Larger scenarios without vehicles (firefighting, building inspection, etc.) can be run by simply pulling the vehicle frame out of the area, and using the environmental factors to create an immersive environment. The headset that is being created by Intel, called Project Alloy, will be used to allow the virtual and real worlds to blend together. Project Alloy displays real objects that are in close proximity to the user, while displaying the virtual world everywhere else. We will also include realistic sounds, smells, temperature and weather to improve the immersion for the first responders. These are controlled from a DMX control station that can be run by the trainers, or can be automated to sensors in the vehicle. This allows the first responder to see their hands and the real tools that they use in real situations, while being immersed in any scenario. Scenarios that this could train include:
• Extrication from a vehicle
• Extrication from a burning vehicle
• Putting out a vehicle fire
• Routine traffic stops
• Unusual traffic stops
• First person on scene
• Addressing passenger injuries by first person on scene (eg. stop the bleed)
These scenarios allow for quick training on multiple versions of a similar scenario. One of the biggest benefits of using a tangible, reconfigurable vehicle with Project Alloy is that you can immerse first responders in dangerous situations repeatedly, without putting them in actual danger before they encounter these situations in the real world. This also allows us to test new equipment (both hardware and software) in a variety of scenarios before it is used in the real world.
A single frame is positioned in a lab. A headset with computer station pre-loaded with Unity or similar gaming engine is used to instantiate the scenario. The environment comes completely out of the box and assembled, or will be assembled with set of instructions similar to that used for furniture. All environmental components will be designed to mount on easy to assemble telescoping rods or stanchions.
This system is very safe for end-users. Two common problems with using VR for training are tripping hazards and the inability to see real objects (collisions). Intel’s Project Alloy addresses both of these potential issues while still providing an immersive experience. The headset itself is not tethered to a computer; all of the processing is done on-board. The ability for the headset to show real objects in the virtual world means that the first responder can see any real objects that are close to them, and avoid them.
The system will use the Intel Project Alloy headset, which works with scenarios built in Unity. The ability to create virtual environments in Unity means that there are endless possibilities to put the first responders through. The Intel Project Alloy headset shows a completely virtual environment, but uses an integrated Intel RealSense camera (depth sensing) to recognize and display real objects when the first responder gets close enough, about 3-4 feet. These real objects that the first responder needs to interact with can be overlaid with matching virtual objects so that the first responders can see them from anywhere in the virtual scenario. The motion tracking cameras allow the first responders to physically move around the environment as they would in the real world.
• Reconfigurable Vehicle – this allows the first responder to touch the vehicle, and interact with it with real tools that they would use in real situations.
• Mannequin – the mannequin will have sensors that will allow the first responders to be tracked not only in how they interact with the vehicle, but also how they provide care once extraction has been completed.
• Tools (examples below)
o Crow bar
o First Aid Kit
o Fire Hose (virtual water)
o Flashlight
o Paper and Pencil
o Stretcher
The system can test both physical equipment and virtual interfaces/simulated technologies.
The physical equipment can be brought directly into the scene with the first responder. A virtual, 3D model of the piece of equipment needs to be made so that the first responder can see it, and as soon as they get close enough to touch it, they see the real object, and can interact with it just like in the real world.
Virtual interfaces and simulated technologies can be integrated directly into the virtual scenario. Anything that the first responder interacts with in the scenario can be tracked, so how the interfaces/simulated technologies impact their performance can be tracked.
- Intel Project Allot headset – approximately $3,000, not available yet, developers version available Q4 2017
- Motion Tracking Cameras – approximately $50,000, multiple options available, well know companies include OptiTrack and Vicon cameras. Price depends on size of tracked space.
- Reconfigurable Vehicle – price dependent on size of area.
- Real Objects – price dependent on what objects are needed to be real
- Virtual Environment – approximately $25,000 per environment – this includes the virtual environment, the fusion of the real and virtual objects and the performance metrics that are tracked.
The reconfigurable vehicle can be used to train a wide variety of first responder scenes.
• Extraction from a vehicle
• Extraction from a burning vehicle
• Putting out a vehicle fire
• Routine traffic stops
• Unusual traffic stops
• First person on scene
If a larger scale scenario is required, the vehicle can be removed and the environmental factors can be kept. This would result in a large VR environment, with the first responders being able to interact with real tools while being in “the elements.”
The selected headset works with the Unity development engine (and most likely others). These development engines allow for the creation of any environment for training use. Switching between these scenarios is as simple as opening a different file.
Training scenarios built in Unity (and other development engines) can have performance metrics built in to the scenarios themselves. This allows for real-time feedback. This is something that Design Interactive has created for similar technology (Canon MREAL headset). The system can track anything that the first responder interacts with within the scenario.
