Virtual Reality (VR) is increasingly being used in the area of training. Especially for professions where errors can cost human lives, VR simulation training can be valuable. Therefore, VR is very suited to improve CBRNe training. To maximize the effectiveness of VR training, it is imperative that such a CBRNe VR training is introduced and implemented sensibly. Building on data from workshops with police officers, trainers and other experts in policing, this study aimed to conduct a comprehensive SWOT analysis, to uncover the current Strengths and Weaknesses of VR as well as identify potential future Opportunities and Threats to a successful implementation of VR in CBRNe training. Results showed that the strengths of VR for CBRNe training are, amongst others, the ability to realistically simulate dangerous situations that are difficult to organise in real-life, the possibility for trainees to train regularly and at any location, and the fact that trainings can be recorded and comprehensively reviewed with trainees afterwards. Lack of technology acceptance, economic aspects, incomplete requirements and technical limitations are the main weaknesses of VR. This has resulted in the formulation of a set of 10 important guidelines to successfully introduce and implement VR in CBRNe training practices.
behavior training, CBRNe training, SHOTPROS H2020 project, SWOT analysis, virtual reality, VR guidelines
In recent years, CBRNe risks (Chemical, Biological, Radiological, Nuclear, explosives) have become a global threat and an important safety concern . There is a strong need to thoroughly prepare for adequate responses to accidental or intentional CBRNe disasters or attacks . When managing a CBRNe situation, four phases will have to be dealt with: threat, preparedness, response and remediation . A key part in being able to deal with such situations effectively is the training of all actors involved, such as first responders, law enforcement agencies, paramedics, or the military [1-5].
However, the training needed for responding to CBRNe threats or disasters is often far beyond the training that these actors actually receive, as there is often little extra time, budget or logistic possibilities to organize training for rarely occurring CBRNe incidents . Nevertheless, actors from these different professions might feel more prepared or be more willing and ready to respond to CBRNe incidents if they receive specialized training. For example, a study with paramedics showed that those who recently (i.e., in the prior three years) received CBRNe training were more willing to respond to CBRNe, and that willingness was furthermore positively related to the quality of their training . Willingness and feelings of readiness increased if the paramedic’s training involved simulations and hands-on practices .
Within emergency planning training, and CBRNe training specifically, five different levels of exercise can be distinguished that need to follow in successive order to allow trainees to master each level before moving on to the next . These five levels are: (a) orientation (low-stress informal discussion in group with little or no simulation), (b) drill (coordinated exercise used to test a single specific operation or function, also involving the deployment of equipment and personnel), (c) Table-Top exercises (group analysis of an emergency in an informal, stress-free environment), (d) functional exercises (fully simulated and semi-realistic exercises that test the capability of an organization to respond to a simulated event, but without the actual deployment of equipment and personnel), and (e) full-scale exercises (a simulated emergency event, as close to reality as possible, involving all emergency response functions and requiring the full deployment of equipment and personnel .
There are many reasons as to why one should carry out regular training and exercises, such as (a) being able to test and evaluate plans, policies and procedures, (b) reveal and respond to weaknesses and gaps in these plans, (c) improve individual performance, organization, communication and coordination, and (d) train personnel and make sure they understand and are able to execute their roles and responsibilities [1, 2].
Ideally, effective CBRNe training programs should be as closely linked to reality as possible [1, 3]. However, often such scenario-type training is very limited in time and frequency and not very realistic, decreasing possible learning benefits from such trainings . Large-scale realistic CBRNe simulation exercises are logistically very difficult, expensive and time-consuming to organize, not to mention the ethical, safety and security issues they impose [6-9]. In this contribution, it will be investigated whether Virtual Reality solutions could be an added value to CBRNe training programs, especially for functional and full-scale exercises (see above).
Virtual Reality (VR) is increasingly finding its way in the area of training and education. Especially for professions where errors can cost human lives, such as for surgeons, the military, aviation and policing, the use of VR for simulations and other forms of hands-on training practices is becoming increasingly common . The emergence of innovative new technologies such as VR and the continuous improvement of these technologies continue to open up new potential in computer-based training applications . VR is an especially useful tool to design and set up regular simulation training that can complement more traditional learning methods, such as classroom-based learning and live training simulations [6, 11, 12]. Simulation training in VR offers great advantages over real-life simulation. For one, it is less cumbersome to set up than large-scale real-life simulations. Computer-based simulations allow for trainees to train more regularly (and thus also increasing personal training time as well as training different positions or tasks) and independent of location. Furthermore, simulation training in VR makes it possible to train in dangerous situations which would otherwise pose too much danger for the safety of the trainees or other persons involved and might not even be allowed to train in real-life. It also allows for the inclusion of vulnerable populations (e.g., children, elderly people, animals) or dangerous equipment (e.g., explosive materials) that would, from an ethical point of view, not be allowed in real-life simulation training.
VR allows users to immerse themselves in specific situations and gives them the feeling of being physically there. Realistically simulated crisis scenarios, in which trainees are fully immersed, allows trainees to actively practice the application of their acquired knowledge and expertise while also being exposed to all the perceptual, motor, and cognitive deficits that are also experienced in real-life high-risk situations [13-15]. VR training encourages input, by turning the trainees into active users who must physically and mentally engage to participate. Thereby, VR experiences stimulate tactile, visual and auditory senses and therefore ensures to satisfy a range of different learning styles or types.
Evidence shows that the use of VR is useful when the training domain is complex and difficult to master [16, 17] and when the audio-visual features of the training environment, assisted by haptic feedback, are crucial to the overall training success . This makes virtual environments an ideal solution for training in areas in which the context of the training is not easily available or replicable due to logistical, security or safety reasons . It offers controlled, easily generated environments that allow for repetition of and variety in different training situations, which is needed to master new skills, generalize competencies across situations an adapt better to novel situations .
VR thus demonstrates its ability to radically transform CBRNe training and learning. From traditional classroom environments to extreme training situations, VR reduces investment and increases enrichment across a range of industries. The possibility to train in virtual environments enables completely new perspectives to design and create training curricula and scenarios. Training of dangerous situations, independence of real-world settings or usage of non-allowed equipment are now possible. On the other hand, the measurement of performance and training progress in VR is a game changer for trainers. Training practices can be recorded, analysed and reviewed together with the trainees frame by frame. Virtual environments offer a maximum of experimental control, are easy to iterate and allow for new forms of feedback and performance monitoring.
Regardless of all these potential advantages of using VR environments for CBRNe training, there is a need for guidelines and heuristics in practice and in the introduction of VR systems in CBRNe training in order to avoid errors at an early stage of the development and implementation. To successfully start using VR as a training tool, especially when the use of VR in a certain training domain is not yet well established, requires a well-considered implementation strategy. This is what is currently being developed in the European Union’s Horizon2020 project SHOTPROS (grant No. 833672) .
The SHOTPROS project aims to investigate the influence of psychological and contextual human factors (HFs) on the behaviour of decision making and acting (DMA) of police officers under stress and in high-risk operational situations. This enables the design of better and more effective training programs for police officers to improve DMA performance in real-time critical operations. To achieve this, SHOTPROS is developing a VR solution to experimentally assess the degree to which these factors influence DMA behaviour.
Subsequently the project will develop a HF-rooted training curriculum and a corresponding VR training solution to provide a comprehensive framework for practical training of DMA under stress and in high-risk (DMA-SR) situations. The training will increase DMA-SR performance, which in turn will lead to better and more correct decisions during operations (from several perspectives, e.g., legal, ethical, proportionality, effectiveness), remaining more in control in threatening situations, a decrease in use of force occurrences as well as casualties and collateral damage.
The goal of the present study is to conduct a SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis on data collected from police officers, police trainers, and other experts in the context of the SHOTPROS project, to uncover the advantages and disadvantages of the usage of virtual training environments in safety-critical situations, as well as the opportunities such usage offers and the threats it can also impose. This results in a set of 10 practical guidelines for setting up VR training in the context of CBRNe training.
As part of the SHOTPROS project, six ‘user requirement workshops’ were organized at six Law Enforcement Agencies (LEAs) across Europe: one in Belgium (Brussels), one in the Netherlands (Amsterdam), one in Romania (Bucharest), one in Sweden (Stockholm), and two in Germany (Selm and Berlin). A total of 60 police officers and police trainers took part in these intensive 1.5-day workshops. The workshops comprised of different sessions related to different topics on VR training of police officers in DMA in high-risk and high-stress situations, such as CBRNe situations. Focus groups were organized to discuss current training practices and future training needs of police officers, and a Wants and Needs analysis was conducted to uncover specific functional and non-functional requirements that participants had for VR training for police officers in the context of DMA-SR.
This resulted in a large data pool that was further enriched by 10 in-depth interviews with a total of 14 experts in the field of police training (e.g., academy leaders), policy making (e.g., heads of police), legal and/or ethical issues (e.g., head of legal department), or any other type of expertise they considered relevant (e.g., instructors from Special Forces).
On the data gathered from the workshops and interviews, a SWOT analysis was conducted. A SWOT Analysis is a strategic planning framework and is therefore an important tool for situation analysis that helps to identify organizational and environmental factors. The goal of the SWOT analysis is to identify what are the strengths of using VR as a training medium and what are currently its weaknesses. From this, it is also possible to determine what opportunities are still open for VR training and how we can use the strengths of VR to create more and better opportunities. Finally, knowing the weaknesses of VR as a means for police training, will also help identify the possible threats that may inhibit wide-spread and effective use of VR as a training medium.
Coming to the SWOT analysis was an exploratory process. The data consisted of all the transcripts from the workshops and the summaries of the observers and facilitators of the workshops. Transcripts of the interviews were also included in the analysis. All data was pooled, analyzed and coded into the four categories of a SWOT: Strengths, Weaknesses, Opportunities, and Threats. Five experts in the fields of technology experience and VR, management and business, and criminology first categorized all the data separately in one of the four segments of the SWOT matrix. Their categorizations were then thoroughly discussed to come to the final SWOT analysis.
Based on this SWOT analysis, an extensive set of practical guidelines was created by the experts for the successful development, set up, and implementation of effective VR training programs. Next, each of the five experts ranked the guidelines in terms of importance. These rankings were then discussed and a final set of the top 10 guidelines was derived and will be presented in this study.
3.1 SWOT analysis
To uncover the advantages and disadvantages of the use of virtual training environments in safety critical situations a SWOT analysis of the gathered knowledge and data was done.
The first advantage of using virtual training environments that was mentioned by many of the police professionals is that it allows for individualized training trajectories. Contrary to real-life training settings, where trainee performance is usually evaluated based on quick, subjective observations, VR offers the possibility to automatically record all the relevant metrics to evaluate training performance in each session, calculate the expected future performance, and as such make the learning progress measurable and plannable. Standardized quantitative parameters and statistics can be extracted by the VR system, such as usage of tactical weapons/devices, movement of the trainees in VR, distance measurement to objects and persons involved, object recognition rate, time to complete a task, or number of shots fired/missed. Furthermore, bio signal indicators can be used, such as breath rate, heart rate, or body temperature. It is, however, important that the metrics used to evaluate the training performance are always determined based on the specific training objectives. Therefore, metrics can differ depending on the concrete training task or VR setting. Another strength of using VR for training is that all the actions of each trainee are recorded, even if they are training together, and can be reviewed in full after the training exercise.
On an individual level this can help to get a more precise view of one’s ability level and to set better, more realistic goals. Team process feedback for virtual teams has shown to have a positive effect on the motivation of less motivated participants and overall has a positive effect on team performance .
Secondly, the prospect of more personalized training sessions was often put forth by many as a key feature of VR training. Virtual training programs can be adapted to the trainee in real-time in terms of difficulty level, based on the strengths and the weaknesses of the trainee. In coordination with the trainer, focus points can be set, which are consequently trained interactively with the trainee. This results in a semi-automated process, where the trainers are relieved of the duty to have to do the trainings themselves. the trainings themselves, thereby freeing resources which can instead be allocated to monitoring the progress and strategically setting new goals and focus points for their trainees.
Another strength of virtual training is seen in the increased frequency of training sessions resulting in more repetitions, which leads trainees to learn and retain their knowledge better. Retention is further strengthened by the immersion and the accompanying increase in emotional affect . In a real-life scenario, trainings are laborious and costly, which limits the number of training sessions per trainee. A tactical training in the military for example, needs to be built up, and can be used by one training group at a time. In VR, multiple groups could train in parallel sessions, each one personalized to the respective group. As VR training can also be done from home, costs in transport, catering and housing for participants of a training event can be cut drastically, whilst the trainees whilst the trainees can train more often.
Lastly, VR training opens up possibilities for debriefing the trainee and reporting the results of a session in a way that would not be possible otherwise. By means of a dashboard, the trainer has an overview of the key performance indicators, past and present, as well as the possibility to jump directly to a time of interest during the training. By replaying key scenarios and adding to that the relevant metrics, trainees can learn more efficiently from their actions, by getting a more complete view on what went well, what did not, and, most importantly, which specific measures need to be implemented in order to improve their performance.
Besides the advantages that virtual training environments offer to police professionals, there were also criticisms that need to be addressed. As VR is still a growing technology, the dependency on the suppliers of the technology is high. Certain desired features of a training may not be implemented yet, as the manufacturers simply do not offer the required technology needed for a feasible price. Economic aspects are furthermore an issue, as the initial cost of acquiring the hardware and software for virtual trainings, as well as the costs of maintenance are considerable. However, these costs are relative, as a VR scenario, once fully developed, can be reused at any time (whereas real-life simulation training cannot).
Taking it a step further, the lack of overarching norms and standards in hardware and software make it difficult to choose what system to buy in the first place. Trainings developed in one engine often cannot be imported into another one, thereby limiting the exchange rate of content between and within organizations. The field of VR research still lacks a sound theoretical basis, as well as technical standards, that are required for growth of a given field . This makes it hard for decision-makers within the police to choose a system, and to justify the purchase to their superiors.
Adding to that, the acceptance of modern technology in traditionally face-to-face environments like police training can vary from employee to employee within a unit. Additionally, if the trainer is not convinced of the virtual training method, the quality of the training will suffer. Resistance to new technology is not a new phenomenon, especially coming from older employees within a company . Even from within our own sample came unimpressed, resistant voices, that rejected the use of VR altogether.
The success and effectiveness of a virtual training is also dependent on the quality of the requirements put forth for the development. If there are features that are missing, or that are only formulated vaguely, the training development becomes obscure and the quality of the resulting product will be questionable at the least. Especially when there is no designated position concerned with modern technology like VR, a lack of understanding of the possibilities and issues with the technology can lead to problematic decision-making regarding acquisition, use and development of virtual training systems.
A promising vision for the future coming out of the workshops is the implementation of large-scale simulations with multiple teams from different disciplines, which together represent the network of professionals associated with certain safety-critical events. For example, different agencies like the police, the military, first responders and nuclear, biological and chemical (NBC) disposal technicians are all involved in a scenario, probably coordinated by different people. A training like this would be high in realism, as it is a holistic representation of reality, where sources of error might surface, which would not be found if every unit just optimized their own activities. This is listed as an opportunity because it requires many different institutions to cooperate and is therefore an external factor not directly in control of the agency that is adopting VR training. Maybe a central VR coordination office could help as an overarching institution in charge of making these large simulations possible.
Right now, because of pragmatic and economic reasons, the trainees are the only agents inside the VR simulation besides the trainer. They interact with NPC’s (non-player characters) to simulate the human aspect of a certain scenario. NPC behavior does not yet compare with real human behavior and interaction, which often is a critical factor in CBRNe situations. Therefore, an opportunity for the future would be human agents within the simulated learning environment, playing the role of civilians, hostages, etc., which formerly were simulated NPCs. This would further add to the realism of the training, as well as train interpersonal skills of the trainees, which can be determinants of success of operations.
Furthermore, machine learning opens up many opportunities for better trainings. Based on the recorded performance metrics data of the trainees inside the VR learning environment, learning algorithms can be applied to find new strategies and tactics, which might result in completely new concepts. Learning antagonists would be another opportunity of machine learning, forcing the trainees to continuously adapt their strategies, preventing plateauing, keeping learners engaged and motivated.
As data are the basis for the success of machine learning algorithms, more sources of data collection could be integrated in the future. Examples would be eye-tracking, physiological measures and movement tracking, resulting in a multi-layered perspective on performance and experience in the scenarios. Cutting-edge technology like haptic feedback suits could further increase realism and immersion of the virtual environments, which leads to more ecologically valid measurements .
Lastly, virtual trainings can be applied in a multitude of areas, like tactical training, behavior training, law and regulations training, and many more. Those areas of application represent potential markets that can be expanded to in the future. Also, this diversity will be beneficial for developing new trainings, as competition and cooperation accelerates growth.
As learning environments in VR have game-like characteristics (repeatability of scenarios, different difficulty levels, simulation without real consequences) there is the threat that trainees might not take their VR training seriously. To avoid that, a clear communication about the effectiveness and real-world applicability of the technology is key to convince every trainee of the advantages and the validity of the training method.
Also, training in VR requires the trainer to know how to handle the technology, so that technical problems do not stand in the way of the learning experience. After-action reviews or the selection of different scenarios and their difficulty level will take some learning on the trainer’s side and might be a threat to a smooth session when the trainers themselves are not well trained. ‘Train-the-trainer’ programs therefore are a necessity for the program’s success.
Training data from professionals in safety-critical occupations is sensitive information. For example, should terrorists get a hold of police training data, they could adapt their strategy to blind spots in the training, possibly compromising the safety of civilians. Data protection of training results is therefore crucial, and one of the biggest threats to using virtual training environments in this context. Additionally, privacy concerns might be raised by the trainees themselves, as every move and gaze can be tracked inside VR. In a study by Kupin and colleagues, the researchers could identify the individual participants 90% of the time only by comparing the 3D trajectories of their hand movements with a library . This only illustrates that anonymity and privacy in VR is not an easy thing to accomplish.
Another threat is the underestimation of follow-up costs with a virtual learning environment system. Technical devices break and need to be repaired or replaced, trainings must be extended and adapted to new needs by a development team and old training material might have to be digitalized first – all of which adds to the initial cost of acquiring a virtual training system.
3.2 Practical VR guidelines
Based on guidelines derived from the SWOT analysis, and the prioritization of these guidelines by a team of five experts, the top 10 guidelines for the development and implementation of VR training for police officers are presented.
3.2.1 Enable efficient scenario control and manipulation for trainers
Live intervention by trainers during scenarios is crucial to facilitate individual and personalized training. This of course requires trainers to have access to a tool that enables them to intervene during live training exercises in VR. With such a tool the trainers should be able to easily change and manipulate the training scenario, the situation and the settings of the scenarios in VR during runtime according to the trainees’ needs. Depending on the actions and performance of the trainees, new events or stressors can be introduced during the training, new tasks can be added for the trainees to complete, or the difficulty level can be raised or lowered along the way.
With such a live intervention tool, the trainer or the system operator can also control the simulated persons in the scenario, start corresponding reactions and thus make them appear intelligent and reactive. Together, these tools will result in a more engaging, interactive learning experience, tailored to the trainees' needs.
3.2.2 Develop a system for multiple-user interaction
Especially considering CBRNe incidents usually involve the deployment of and the coordination between many actors from varying professions (e.g., police, coordination team, local authorities, paramedics), the virtual system must enable a multiple user interaction within the training. Even if the VR training is used to only train a smaller segment of such a coordinated action, a minimum of three persons interacting at the same time is a must. Furthermore, as coordination and communication are so crucial during CBRNe disasters, communication means within the VR training should be available for all trainees to communicate with each other and with the coordination office. A multi-user interaction system also helps the trainer to communicate with each trainee separately or with different sub-groups by use of different communication channels. Especially when working with larger groups this is crucial, as communication tends to get chaotic otherwise.
3.2.3 Put effort in the after-action review dashboard
As mentioned in the SWOT analysis, the possibility of the after-action review dashboard is one of the most relevant features of VR training. There is great potential in VR when it comes to providing training feedback to trainees. During real-life simulation scenarios, trainers can often only rely on subjective and sometimes limited observations (especially in larger-scale simulations) for their feedback. Therefore, such feedback is often outcome-based and general (e.g., was the team able to successfully complete a task or reach a goal?). VR, on the other hand, also allows to provide detailed feedback on the actual process and performance of the individual trainees. It can keep track of the performance of each individual trainee throughout the entire exercise, it can use scripts to automatically record data on certain parameters (e.g., distance kept, number of shots fired, number of hits, elapsed time before taking a certain action), and it could even allow for the recording of physiological measurements (e.g., heart rate variability) or eye-tracking.
Providing a well-designed, elaborate after-action review dashboard can be a game-changer for training programs. Trainers should be given all the tools to provide feedback on the individual level and on the coordinated team-level. They should be able to jump to certain highlights or mistakes made during the training, to re-play it for the trainee(s) and immediately illustrate their feedback with relevant footage of the training. Trainers should also be offered the tools to interpret the recorded parameters correctly.
3.2.4 Create “Train the Trainer” concepts as a fundament
In order to exploit the advantages of virtual environments, the training of trainers on how to work with the VR training tool is highly relevant for success. Our studies have shown that this point is often underestimated. It is recommended to develop a concrete training plan to specifically train the trainers in using such a VR training tool to its full benefit. Trainers should for example be educated on how to set up the system, how to develop new or make changes to existing VR scenarios, how to start a training, how to interpret different parameters and make full use of the after-action review dashboard, and how to do basic troubleshooting. Ideally, a full training course should be available for each trainer before starting to use the VR system for their courses, as well as a clear manual on all the functionalities of the system. Similarly, if updates or additional functionalities are added to the VR system, trainers should follow a short course getting acquainted with the new functionalities. It would also be highly advisable to organize yearly refresher courses for the trainers, to evaluate whether they are still using the VR training tool properly and to guarantee that they continue to make maximal use of all the possibilities of the VR training tool.
Besides providing the trainer with the required information and education, it is also advisable to allow the trainers to explore virtual reality and the training environments within it on their own. That way they get a natural feel and understanding of the technology and the experience of VR, additionally to structured training programs.
3.2.5 Implement real-world devices to support immersion
Figure 1. Tactical Belt for VR - from the SHOTPROS Project (developed by RE-liON)
Data from the workshops with 60 police officers and trainers from five different European countries revealed that when police officers are training real-life or virtual scenarios, they find it very important to be able to use the equipment they would normally also wear and use in the field (such as communication devices, torch, weapons). Furthermore, based on the analysis of the conducted interviews one of the most highly ranked features for a future VR system was the usage of real-world equipment in the VR training. Therefore, from an end user perspective, it is a must to integrate these devices in the VR, but also to create them tangibly (and not only virtually). In the SHOTPROS Project, for example, a tactical belt was developed exclusively for use in VR by the RE-liON team (see Figure 1). By wearing a physical tactical belt during a VR training session, real muscle memory is built up and psycho-motor skills are thereby trained alongside the cognitive and affective skills. This makes for a far more realistic training, as the actual situation is mimicked closely on different levels of sensory input (visual, auditive and haptic).
3.2.6 Allow free movement in the virtual environment
The free movement within the virtual environment was mentioned as a key aspect in the data from the user workshops. Therefore, it is strongly recommended to use a VR system which is not wired to a central hardware. A free movement enables the usage of a broad range of scenarios and will reduce the risk of technical limitations. This is of special importance in the case of group trainings, where the trainees perform their tasks in the same space, but in a simulated environment. Wires would be more than an annoyance in this case, as tripping over them would be a major health hazard for the trainees as well as possibly leading to expensive damages to the hardware. Investing in wireless gear is therefore advantageous, though costlier at the beginning. The return of investment however is the safety of trainees and reduced maintenance costs.
3.2.7 Combine real-world training with VR training
VR training should be complementary to other training methods. It cannot and should not replace, for example, desk-based learning, role-playing and real-life simulations. Furthermore, VR training is not suited for all training objectives. One of the key findings from the data and SWOT analysis is that there are certain training domains for which VR is a very suitable tool (e.g., tactical training, communication and cooperation) and other domains for which it is not, due to current limitations in VR technology (e.g., accurate shooting training, hands-on combat training).
Therefore, VR should be viewed as an additional learning method that, together with other methods, can benefit the learning efficiency of the full training program. VR should only be used if it facilitates equal or higher learning efficiency for a certain training objective compared to the training methods that are already in use.
3.2.8 Put effort in maximizing the ecological validity of training scenarios
VR has the capability of improving the ecological validity of training scenarios by enhancing the realism of the training situations through immersive virtual environments [28-30]. Ensuring a strong feeling of being ‘immersed’ in the scenario is critical in developing an effective virtual environment . A trainee should let go of the idea that the stimuli presented in the virtual reality are in fact not real, i.e., suspension of disbelief . A virtual environment is ecologically valid when it induces cognitive, emotional and affective reactions in trainees that are similar to the reactions they would exhibit in similar real situations . Therefore, relevant human emotions and behaviors should be triggered in a similar way as they would in the real world.
This also includes investing in sufficient sensory feedback to the trainees during the training. In developing training scenarios, the use of haptic interactions should be integrated, for example by simulate the recoil when firing a weapon, or using mildly averse vibration stimuli when trainees get injured in the VR scenario. Furthermore, also consider the density of objects and materials in a scenario. If an explosive goes off or shots are fired, would a person be sufficiently protected when hiding behind a car or would they get severely injured? All these elements should also be taken into account when developing a training scenario.
3.2.9 Continuous ethical reflection is advised
When developing a VR training system, a thorough reflection on ethical and legal issues related to the training content, the training method, and the safety of the training tool is important.
Scenario-based training is not a game and should always be about learning appropriate and proportionate behavior given the situation at hand. Therefore, it is not a means to deliberately perform unethical, illegal or disrespectful behavior or play. Trainees should be refrained from doing things in VR just because it is possible (e.g., starting to shoot at each other or at bystanders), even if it is just ‘for fun’. Of course, it should be possible to make mistakes in a training scenario and learn from it.
VR offers endless possibilities in designing and developing a variety of training scenarios. This offers a lot of new opportunities for training but can also contain the risk of intentionally or unintentionally stigmatizing or stereotyping certain groups, based on their ethnicity, culture, religion, gender or appearance. Trainers should have the possibility to sufficiently diversify and vary appearances of characters within training scenarios. Scenario editing programs should come with a large and diverse character database to choose from. Trainers should also keep guard to not systematically select stereotypical characters for certain roles in the training scenario.
The scenarios presented to trainees and the stressors that are embedded in the scenarios, should always be directly relevant to achieving the desired training objectives. It cannot be that trainees are unnecessarily exposed to the most horrific situations, just because VR makes it possible. The level of exposure to stressful and risky training situations should always be proportionate to the requirements to meet a certain training objective.
3.2.10 Be transparent about data storage and protection of privacy
As a VR training tool allows the recording of the entire training of each trainee along with the personal physiological measurements and performance metrics, an important question is what to do with all this data? There are many options when it comes to data storage of training data from the trainees. Training data could be saved until after the trainee receives feedback on their performance in a given training scenario, and the data can then be deleted. Training data could be stored in a general agency-wide database to be used as examples in classroom-based training. The data could also be stored for each trainee separately, enabling the trainee (and perhaps the trainer) to monitor performance and progress across training sessions.
This guideline does not imply that one option is more advisable than the other. This choice highly depends on the goal of the VR training and regulations within a specific law enforcement agency. However, whatever choice is made in the end, it is important to thoroughly discuss these issues before developing the VR training system and it is always necessary to fully inform the trainees (and trainers) on which data will be stored (and get their informed consent), whether or not the data will or needs to be anonymized or pseudonymized, for how long it will be stored, who will be able to access it, for which purposes it is stored and can be used, and how the data will be protected.
This study highlights the vast potential of VR for implementation as a training tool within CBRNe training. Preparing actors from various disciplines for incidental or malicious CBRNe incidents is crucial, yet often difficult to organize. To train actors well, functional and full-scale simulation exercises are a must, but organizing such exercises in real-life is time-consuming, expensive, logistically complex, and – most of all – subject to profound safety and security issues. Using VR for CBRNe simulation training could be the answer to this problem.
The SWOT analysis, based on in-depth qualitative data collected in 6 extensive workshops with police officers, trainers and other experts, clearly identified many strengths and opportunities of the VR technology for CBRNe training. The main strengths are that VR offers the possibility to track each individual trainee throughout the entire training, record and measure several parameters (relevant for the specific training goals) and use footage of certain trainee actions in the after-action review. Unlike real-life simulation exercises, VR scenarios can be re-used repeatedly, modified easily by adding, changing or removing objects, characters, or events, and can be trained anytime, anywhere. Furthermore, VR allows for immersion of the trainees in a setting that triggers similar emotional, perceptual and cognitive states as real-life situations would. All these advantages make it worthwhile to further investigate possible ways to incorporate VR training in the CBRNe training curriculum.
Of course, VR also has several disadvantages or weaknesses that complicate easy implementation of VR systems. For example, designing a system that meets the specific training needs can be very cumbersome, expensive, and confusing, especially if you don’t have extensive technological knowledge. It should be very clearly communicated what the system should be capable of, and how exactly it should be implemented. Furthermore, VR is still a growing technology, and even though it already offers so many possibilities, it is still in full expansion. That means that there are technological limitations to what is currently feasible or possible. However, end users often have unrealistic expectations and demands concerning what type of system and functionalities they want. It is thus crucial that there is a constant feedback loop between the clients and the developers.
This study concludes that, if developed and implemented carefully, VR can be of high added value in any CBRNe training. However, when there is a lack of knowledge about and experience with VR or other innovative technologies in an organization, implementing such a system in a training program can be difficult. If done improperly, a VR training system might not be used at all because nobody knows how to work with it. Also, it might not provide effective training because the system doesn’t meet the requirement criteria or supports the desired didactical approach.
For future work we recommend focusing further on the topic of measuring the effectiveness of VR training in comparison to real world training. Therefore, we have also planned a study to directly compare VR training to conventional training. We will re-create a real-world training 1:1 in VR and use it in several training sessions. One group will train with the VR system and the second group will train using the conventional methods. With the help of a comparative results analysis, the progress of the two groups will be compared after the training sessions in order to examine the effectiveness of VR training. Relevant indicators will also be identified that can be used in future comparative studies to better investigate the effects of VR training.
Based on the SWOT analysis conducted in this study, we put forth 10 guidelines we deem important to take into consideration when contemplating developing a VR-based CBRNe training system. Of course, these guidelines are not exhaustive, as there are many elements unique to every considering institution one should critically reflect upon in the entire development, set up and implementation process. Nevertheless, these 10 guidelines do provide a good basis that covers a range of aspects in such a VR system, from the development of VR environments and hardware, features during training, the creation of training scenarios, to the after-training debriefing.
These guidelines are an intermediate product of extensive ongoing research within the SHOTPROS project. The SWOT analysis and the guidelines are based on qualitative data from end user workshops. In the meantime, several experimental studies have been conducted, providing a multitude of data, including more than 899 qualitative interviews about the experiences of participants in the studies with the VR system, as well as experience measurements. This data is currently being processed and analyzed and will allow for further additions and/or refinements to the SWOT analysis and the guidelines.
In a next phase of the project, the guidelines will further be empirically validated through several Human Factor-based studies and expert reviews from experts within several CBRNe domains. At the end of the project, we aim to provide CBRNe trainers and policymakers with evidence-based guidelines, focusing on several aspects such as the design of a VR system, the technical requirements for such a system, features that should be included for maximal learning profit, and guidance within the purchase process. This way, we aim to assist in and promote the successful and EU-wide implementation of an effective CBRNe VR-supported training system.
This work is supported by the European Commision’s Horizon 2020 Research and Innovation Programme (Grant number: 833572).
We thank Quynh Nguyen for their valuable comments to the SHOTPROS project, study design and analyses and all involved police officers for their valuable feedback and work in the user workshops. We also acknowledge the great support by Jakob Uhl to finalize this publication.
 Malizia, A., 2013/2014 CBRNe Masters Group. (2016). Disaster management in case of CBRNe events: An innovative methodology to improve the safety knowledge of advisors and first responders. Defense & Security Analysis, 32(1): 79-90. https://doi.org/10.1080/14751798.2015.1130319
 Coleman, C.N., Bader, J.L., Koerner, J.F., Hrdina, C., Cliffer, K.D. (2019). Chemical, Biological, Radiological, Nuclear, and Explosive (CBRNE) science and the CBRNE Science Medical Operations Science Support Expert (CMOSSE). Disaster Med Public Health Prep, 13(5): 995-1010. https://doi.org/10.1017/dmp.2018.163
 Rothbacher, D. (2017). Combining theoretical education and realistic, practical training: the right approach to minimize CBRNe risk. Cyber and Chemical, Biological, Radiological, Nuclear, Explosive Challenges, pp 327-343. https://doi.org/10.1007/978-3-319-62108-1_16
 Beyramijam, M., Khankeh, H.R., Farrokhi, M., Ebadi, A., Masoumi, G., Aminizadeh, M. (2020). Disaster preparedness among emergency medical service providers: A systematic review protocol. Emergency Medicine International. https://doi.org/10.1155/2020/6102940
 Anan, H., Otomo, Y., Kondo, H., Homma, M., Koido, Y. (2016). Development of mass-casualty life support-CBRNE (MCLS-CBRNE) in Japan. Prehospital and Disaster Medicine, 31(5): 547-550. https://doi.org/10.1017/S1049023X16000686
 Göllner, J., Peer, A., Meurers, C., Wurzer, G., Schönauer, C., Kaufmann, H., Bösch, C. (2019). Virtual reality CBRN defence. Meeting Proceedings of the Simulation and Modelling Group Symposium, 171: 1-25.
 Jaspaert, E., Rato, D., Prada, R., Paiva, A., Vervaeke, G. (2018). A virtual platform to train cross-national police teams in team collaboration and police interviewing. European Law Enforcement Research Bulletin, 4SCE: 167-175. http://188.8.131.52/index.php/bulletin/article/view/322
 Bacon, L., MacKinnon, L. (2012). Using virtual and augmented reality to create realistic training events. Compass: The Journal of Learning and Teaching at the University of Greenwich, 6: 61-70. http://184.108.40.206/index.php/bulletin/article/view/322
 Salas, E., Wildman, J.L., Piccolo, R.F. (2009). Using simulation-based training to enhance management education. Academy of Management Learning & Education, 8: 559-573. https://doi.org/10.5465/amle.8.4.zqr559
 Fowlkes, J., Schatz, S., & Stagl, K.C. (2010). Instructional strategies for scenario-based training: insights from applied research. SpringSim ’10: Proceedings of the 2010 Spring Simulation Multiconference, San Diego, California, USA. https://doi.org/10.1145/1878537.1878571
 Lamb, R., Lin, J., Firestone, J.B. (2020). Virtual reality laboratories: A way forward for schools? Eurasia Journal of Mathematics, Science and Technology Education, 16(6): em1856. https://doi.org/10.29333/ejmste/8206
 Haskins, J., Zhu, B., Gainer, S., et al. (2020). Exploring VR training for first responders. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Atlanta, GA, USA, pp. 57-62. https://doi.org/10.1109/VRW50115.2020.00018
 Andersen, J.P., Papazoglou, K., Arnetz, B.B., Collins, P. (2015). Mental preparedness as a pathway to police resilience and optimal functioning in the line of duty. International Journal of Emergency Mental Health and Human Resilience, 17: 624-627. https://doi.org/10.4172/1522-4821.1000243
 Andersen, J.P., Gustafsberg, H. (2016). A training method to improve police use of force decision making: a randomized controlled trial. SAGE Open. https://doi.org/10.1177/2158244016638708
 Marquardt, N., Krämer, L., Schürmann, V. (2019). Mental preparation strategies and firefighter’s performance under stress. International Journal of Safety and Security Engineering, 9(4): 332-343. https://doi.org/10.2495/SAFE-V9-N4-332-343
 Huang, H.M, Rauch, U., Liaw, S.S. (2010). Investigating learners' attitudes toward virtual reality learning environments: based on a constructivist approach. Computers & Education, 55(3): 1171-1182. https://doi.org/10.1016/j.compedu.2010.05.014
 Bossard, C., Kermarrec, G., Buche, C., Tisseau, J. (2008). Transfer of learning in virtual environments: A new challenge? Virtual Reality, 12: 11-161. https://10.1007/s10055-008-0093-y
 Abate, A.F., Guida, M., Leoncini, P., Nappi, M., Ricciardi, S. (2009). A haptic-based approach to virtual training for aerospace industry. Journal of Visual Languages & Computing, 20(5): 318-325. https://doi.org/10.1016/j.jvlc.2009.07.003
 Grabowski, M., Lepak, G., Kulick, G. (2009). Collaborative technology impacts in distributed learning environments. Innovative Mobile Learning: Techniques and Technologies. https://doi.org/10.4018/978-1-60566-062-2.ch007
 Martin, G.A., Schatz, S., Bowers, C., Hughes, C.E., Fowlkes, J., Nicholson, D. (2009). Automatic scenario generation through procedural modelling for scenario based training. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 53: 1949-1953. https://doi.org/10.1177/154193120905302615
 Geister, S., Konradt, U., Hertel, G. (2006). Effects of process feedback on motivation, satisfaction, and performance in virtual teams. Small Group Research, 37(5): 459-489. https://doi.org/10.1177/1046496406292337
 Chittaro, L., Buttussi, F. (2015). Assessing knowledge retention of an immersive serious game vs. a traditional education method in aviation safety. IEEE transactions on visualization and computer graphics, 21(4): 529-538. https://doi.org/10.1109/TVCG.2015.2391853
 Garrett, B., Taverner, T., Gromala, D., Tao, G., Cordingley, E., Sun, C. (2018). Virtual reality clinical research: promises and challenges. JMIR Serious Games, 6(4): e10839. https://doi.org/10.2196/10839
 Widagdo, P.P., Susanto, T.D., Setyadi, H.J., Wardhana, R., Taruk, M., Pakpahan, H.S. (2021). The influence of user generation differences on individual performance in using information technology. Journal of Physics: Conference Series, 1803(1): 012031. https://doi.org/10.1088/1742-6596/1803/1/012031
 Parsons, T.D. (2015). Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences. Front. Hum. Neurosci, 9: 660. https://doi.org/10.3389/fnhum.2015.00660
 Kupin, A., Moeller, B., Jiang, Y., Banerjee, N.K., Banerjee, S. (2019). Task-driven biometric authentication of users in virtual reality (VR) environments. International Conference on Multimedia Modeling, Thessaloniki, Greece, pp. 55-67. https://doi.org/10.1007/978-3-030-05710-7_5
 Kim, D., Jung, S. (2020). Using eye-tracking technology to measure environmental factors affecting street robbery decision-making in virtual environments. Sustainability, 12(18): 7419. https://doi.org/10.3390/su12187419
 Laciok, V., Bernatik, A., Lesnak, M. (2020). Experimental implementation of new technology into the area of teaching occupational safety for industry 4.0. International Journal of Safety and Security Engineering, 10(3): 403-407. https://doi.org/10.18280/ijsse.100313
 Van Gelder, J.L., Otte, M., Luciano, E.C. (2014). Using virtual reality in criminological research. Crime Scene, 3(10): 1-12.
 Toet, A., van Schaik, M.G. (2012). Effects of signals of disorder on fear of crime in real and virtual environments. Journal of Environmental Psychology, 32(3): 260-276. https://doi.org/10.1016/j.jenvp.2012.04.001