Wearable technology on the battlefield: Display technology changes the future soldier

    According to foreign media reports, for decades, augmented reality displays - which can expand the world in front of the user and provide various useful information, such as the attacker's body temperature and the speed of the incoming missile - have appeared repeatedly in science fiction movies. Thanks to recent technological advances and cost reductions caused by emerging consumer markets, the functions of augmented reality (AR) and virtual reality (VR) technology are increasingly catching up with the vision of Hollywood movies.

    In the defense field, which requires various complex operations and training, VR and AR tools are becoming increasingly useful and important.

virtual reality technology

    Virtual reality technology has actually been used by the defense industry as a training tool for some time. As early as the 1980s, the US military enlisted experts from the game company Atari to develop a training system for its Bradley fighting vehicles based on the company's shooter game Battlezone.

    While these early systems were fairly rudimentary and not widely used, advances in processing power and display technology over the past decade have made this technology popular in the military. In the UK, the Royal Air Force has been using advanced VR training systems to train helicopter crews for several years.

Crew Training System

    Developed by British company Virtalis, the system consists of a head-mounted display, a wooden surround that simulates the interior of a helicopter, and a camera-based tracking system that monitors the position of the user's head to give them a precise location in the virtual world.

    The technology allows trainees to practice a range of different scenarios before their first helicopter operation. According to Virtalis managing director David Cockburn-Price, it has helped the RAF use its training resources more efficiently. “They no longer have to waste time in the air giving students basic training because they have already learned it on the ground,” he explained. “The RAF has also seen a significant reduction in training failure rates since deploying the system.”

    The Virtalis system is quite high-end, and while it can help save on flying costs, it is still expensive: the monitor alone costs between £15,000 and £30,000, according to Cockburn-Price.

    At the same time, researchers are increasingly looking to cheap, readily available technologies from other fields for solutions.

    In the UK, one of the main organisations actively encouraging researchers to do this is the Defence Science and Technology Laboratory (DSTL), which funds a number of projects in this area.

    Recently, Plextec, a British electronics industry consulting company, released a virtual reality system designed to train soldiers in medical emergencies. The system is based on the commercial Oculus Rift virtual reality helmet. The project was funded by the DSTL's Defense Enterprise Center (CDE), which allows trainees to experience the scene of making difficult clinical decisions under fire.

Plextec 's VR tool uses Oculus Rift headsets to train soldiers to handle medical emergencies

    The technology – which will cost just a few hundred pounds once it can be mass-produced – is currently being tested by the Ministry of Defence and could start to be used by the military in a few years, according to Collette Johnson, medical business manager at Plextec.

    Exploring the potential use of commercial off-the-shelf (COTS) products is also one of the main goals of the DSTL Synthetic Environment program, which is helping to develop a range of virtual reality training systems based on existing technologies. One prominent example is an experimental driving simulator for the latest version of the Warrior infantry fighting vehicle, which is expected to enter service later this year.

    Caroline Shawl, technical partner on the project, explains that with the new version of Warrior, DSTL wanted to see if simulation technology could be used to train tracked vehicle maneuvers. According to her, the simulator, based on an off-the-shelf mobile platform and image generation system, was developed on time and successfully demonstrated that training on the simulator is just as effective as training on the real vehicle.

    This type of training has not only changed the MoD's requirements for future training systems, but, according to Shore, it has also changed people's perception of the purpose of simulators. Her colleague, Ian Greig, a scientist in the technical group, adds that there is a growing recognition that simulator technology has become more effective while becoming cheaper. "In the past, we might have spent £50 million on two high-fidelity high-speed jet simulators to train pilots," he says. "Now we can buy 25 of the same equipment for £1 million each and string them together to give trainees much better real-life flight preparation."

Impact of the gaming industry

    Of the industries that could benefit the most from helping to drive down the cost of analog technology, the one that could be most impacted is gaming, though Shore stressed that the resulting games may not necessarily be the ones you want.

    “The level of realism in games can be distracting,” she said. “You want to get training effect from training, but it’s easy to be distracted by the images on the screen. But the most important elements for us are the GPU, the controller, the PC, and the performance of these devices.”

Artist rendering of Ubisoft 's Ghost Recon : Future Soldier

    These advances will also come in handy in DSTL’s latest project, which seeks to improve the performance of computer-generated “entities” (such as ground troops or civilians) used in various virtual training tools.

    Gregg said one of the problems with existing tools is that those entities - whether they are army units or fighter jets - currently have limited autonomy. "We have tools that generate thousands of entities, but we need a lot of people to control them," he said. "You can let them do what they want for about five minutes, but then you have to take control of them and make corrections. You might need one controller for every 50 entities."

    Therefore, one of the main goals of DSTL's future SCORE (Simulated Composition and Representation of Natural and Physical Environments) project is to increase the number of entities and reduce the number of people controlling them.

    Gregg said it was a challenging problem. "How do we make the whole city in the game perform its duties? How do we make the civilian population do what they normally do and react to the military activity going on around them? How do we reproduce the dynamics of the enemy? How do we reproduce the insurgents?"

    The project has not officially started yet, so Greg and Shore can only guess how all this will be achieved. However, Shore pointed out that they will study many areas for improving the interface, allowing controllers to control more battlefield situations and embed higher levels of artificial intelligence.

Augmented Reality

    The tools developed through SCORE will have a significant impact on military training, but many believe that the biggest impact on the military may be the rise of augmented reality and virtual reality combat scenarios.

    So far, augmented reality defense applications have basically just been very advanced head-mounted displays worn by fighter pilots.

    Perhaps the most notable of these are the BAE Systems Striker helmet for Eurofighter pilots and the F-35 helmet developed jointly by Rockwell Collins and Elbis, which was recently launched and is considered the most advanced augmented reality helmet ever made.

The F35 helmet is known as the most advanced augmented reality system ever developed in the defense field

    While there are differences in how these helmets operate, at their core they all monitor the exact position of the pilot's head, thereby displaying relevant, accurate information directly in front of the pilot's eyes.

    The latest Striker helmets and F-35 systems have a unique feature: the ability to display images from cameras mounted on the outside of the aircraft, allowing pilots to see the entire structure of the aircraft and obtain an uninterrupted 360-degree view.

Land-based augmented reality system

    The engineering challenges of developing this technology are incredibly complex in themselves, but engineers are now trying to take augmented reality to a new, even more technically challenging area: developing a head-mounted system for the Army.

    "Augmented reality has been around in cockpits and aircraft for a long time," says Dr David Roberts, who leads the development of military action and sensing systems at Applied Research Associates, a US company at the forefront of this emerging field. "But it's just starting to expand to the Army. In an aircraft, you have a lot of resources to draw on, you can plug displays into a big power source, you have computers and you can use a wide range of sensors. Soldiers on the ground don't have those resources, and the focus is just on reducing weight and energy consumption."

The ARC4 system can display useful information directly in front of the soldiers

    Using funding from the Defense Advanced Research Projects Agency (DARPA), Roberts' team has developed a software and sensing package that it hopes will support this emerging soldier technology.

    The system, called ARC4 (for Augmented Reality Command, Control, Communications and Coordination), aggregates inertial sensor, GPS and camera data to record and track the position and orientation of the user's head. With this data, the system's software can pull information from any communications network it is connected to and present it directly to the soldier's eyes.

    Roberts said the system was developed primarily for joint terminal attack controllers (JTACs), who could use it to monitor the location of friendly forces that are out of sight or within range, for example.

    He added that a big advantage of this system over existing tablet-based systems is that troops can stay focused on the battlefield. "When you look down at a tablet, you can't focus on what's going on around you," he said. "Your brain has to make the connection between the 2D information you see when you look down at the map and the 3D information you see in the real world when you look back up. If you look at that information heads up, you're always aware of what's going on around you and you're getting that information right in front of you."

    Roberts stressed that this technology could be integrated into any head-mounted display and is much more advanced than commercially available systems such as Google Glass. "Those commercially available devices do not provide true augmented reality in terms of information presentation, they are distracting and cannot be adjusted based on your location."

The lightweight, full-color QWarrior system will soon be put into use on the battlefield

    Meanwhile, in the UK, BAE Systems is using funding from a DARPA program to develop the Q-Warrior system, a clip-on, full-color augmented reality display for the Army.

    Chris Colston, director of business development for advanced displays at BAE Systems, noted that advances in waveguide displays have made it possible to create lightweight and portable head-mounted devices.

    He explains that the system replaces the complex lens configurations found in normal optics with a waveguide consisting of parallel glass sheets. The image is embedded in this waveguide and bounces forward until it reaches a diffraction grating between the glass sheets. This is how the image is directed to the eye's shuttle box, which the user can see.

    Like the ARC4, the Q-Warrior also includes a tracking package that uses a combination of GPS and inertial sensing technology to measure the exact position of the user's head.

    Both Q-Warrior and ARC4 are currently undergoing trials, and while both systems are nearing completion (Colston expects a production version of Q-Warrior to be available within 18 months), BAE and ARA are continuing to develop capabilities for them.

    Roberts' next challenge is to develop a system that works in areas without GPS. He won't reveal how he will solve this problem, only saying his team has a chance to achieve it. "We plan to add technology that will allow you to carry the system when you enter buildings and are in 'non-GPS' areas."