Virtual reality as safe training for high-risk situations Training specialists in the disposal of unexploded ordnance (EOD) is a long-term and demanding process that today relies on a combination of theory and limited practical training. However, real-life scenarios are logistically and safety-challenging – and often cannot be repeated with sufficient variability. This is where virtual reality comes into play. Researchers from the Faculty of Engineering and Technology of Czech Technical University, in cooperation with engineers from the Czech Army, are creating an environment in which specialists can train entire decision-making processes – from the first contact with a suspicious object to the design of a safe procedure for its disposal. Unlike conventional simulations, the project does not focus on the “mechanics of intervention”, but on the correct assessment of the situation. The key is to recognize the type of ammunition, assess the risks and choose the right procedure. “It’s not about learning one specific intervention, but understanding the entire process – how to identify ammunition, how to safely approach it and how to make a decision in a given situation,” explains Prof. Jiří Žára, Head of the Department of Computer Graphics and Interaction, who is managing the EOD project. Virtual scenarios are not static. Researchers work with procedural generation of the environment, thanks to which situations change and are not predictable. “We want the training to not be based on simply memorizing the scenario, but for the user to have to really think and react to new conditions, similar to real deployment,” adds Dr. David Sedláček, who manages the Virtual and Augmented Reality Laboratory. The system also includes simulations of real tools – such as metal detectors or spectrometers – that enable the “finding” and identification of explosives. Analysis of user behavior also plays an important role: the system monitors movement in the scene and evaluates whether safety procedures have been followed. An important part of the project is also training in working with robots for munitions disposal. “Special EOD robots are expensive and in practice there are only a limited number of them. The virtual environment allows us to train in their control on a larger scale,” says Prof. Žára. “Modern technologies today are fundamentally changing not only the battlefield itself, but also the way soldiers are trained. Virtual reality allows us to effectively simulate situations that would be very time- and financially demanding to train in a real environment. In the field of EOD, I see its great benefit, especially in the training of decision-making processes – that is, in how to correctly identify ammunition, choose a safe procedure and minimize risks. At the same time, it opens up space for wider use, for example in combination with 3D modeling, which can significantly facilitate training and understanding the functioning of individual types of ammunition,” says Lieutenant Colonel Ing. Martin Turek, Chief of the Technical and Information Support Center of the 15th Engineer Battalion of the Army of the Czech Republic. Holographic control room: a three-dimensional overview of the situation The second project, known as HOLO-Swarm, focuses on real-time operations management and decision-making. It is being created in collaboration with the Multi-Robotic Systems Group (MRS) led by Prof. Martin Saska, which has been developing autonomous drones and robotic systems at the Department of Cybernetics at the Faculty of Electrical Engineering and Mechanical Engineering of the Czech Technical University for a long time. Today's systems for monitoring the situation in the field are often based on two-dimensional maps, which have their limits when working with larger amounts of data. Augmented reality may be the solution. Researchers from the Faculty of Electrical Engineering and Mechanical Engineering of the Czech Technical University are therefore developing a system that allows the entire "battlefield" to be displayed as an interactive 3D model directly in front of the user. "Classic maps are flat and working with them is limited. We want to give the user a spatial view of the situation and the opportunity to work with it interactively," describes Dr. David Sedláček. The user uses AR glasses and sees a three-dimensional model of the terrain in front of him, onto which current data is projected. The system displays the movement of units – for example, swarms of drones – their planned routes and current status. “The user can choose what data he wants to monitor – from basic telemetry to detailed 3D reconstruction of the surroundings. The goal is to have the best possible overview of what is happening in the field,” says Sedláček. The system also allows for active intervention in the control. The commander can enter new targets, adjust trajectories or replan the mission and monitor how the changes take effect. Another great advantage is the sharing of data between multiple users. “The same virtual space can be shared by multiple people – whether they are in one place or at different locations. Everyone sees the same situation and can solve it together,” adds Sedláček. The company QuaternAR, which operates in the defense technology segment, is also involved in the implementation of the HOLO-Swarm project. Its software is used in military environments, including the Czech Army, but also in the United States and Israel. “Semi-autonomous robotic assets will form the backbone of defense in the future, especially in situations where a potential adversary has significantly greater human resources. Augmented reality allows for a better understanding of the battlefield situation and a deeper immersion of the operator in the control of these assets,” says Jan Hovora, CEO of QuaternAR. From defense to critical infrastructure.
Although both projects are being developed within the framework of defense research (PRODEF program), their use is significantly broader. “These technologies are not just for the army. We also see their use in the integrated rescue system or in the protection of critical infrastructure, where it is necessary to quickly assess the situation and make decisions,” says Prof. Jiří Žára. Both projects are still in the prototype phase. Researchers are now testing the first versions of the systems and verifying their usability in practice. “We are at the beginning and are only verifying how well these approaches will work in real deployment. Testing with users and further development will be key,” concludes Dr. David Sedláček.