【Original】AR: Bringing cool experiences beyond games to the real world

By Peter Brown, Mouser Electronics

If you ask random people on the street about what virtual reality (VR) is and how it works, most people will tend to answer about the technical aspects of VR, or even equate it with the VR technology used in the TV series "Star Trek: The Next Generation." But if you ask the same question about augmented reality (AR), it's not easy to get an answer. People will talk about gaming technology or the early version of Google Glass, which ended in failure.

The concept of AR has been around for a while, but it has been quietly gaining traction. It wasn’t until the release of Pokémon Go that AR really entered the public eye. Pokémon Go is a video game that combines smartphone imagery with a real-world treasure hunt, allowing players to experience both the real and virtual worlds simultaneously. But what exactly is “AR”? And as an emerging technology, does it have the potential to go beyond gaming and entertainment to become a solution to real-world problems? This article explores a variety of AR pilot projects and applications across industries, including healthcare, law enforcement, education, logistics, construction, and more. You’ll see how AR technology is being transformed into a usage platform that is being put to use in multiple industries—from simple pilot projects to everyday life.

Augmented reality, sometimes called mixed reality, is a technology that combines real-world objects or environments with virtual elements generated by sensory input devices such as sound, video, images, or GPS data. Unlike virtual reality, which completely replaces the real world with virtual objects, augmented reality operates in real time, with objects in the environment being interactive and overlaying virtual displays on the real environment.

As games become more popular, AR technology has fully demonstrated its ability to bring interactive digital worlds into the personal perception of the real world. In this world, digital can show more information about real-world objects, even beyond the objects that are actually visible in the real world. Imagine: when a doctor examines a patient, he can know the patient's physical condition without scans and X-rays; when children learn about molecules, they can interact with molecules; just by looking at a machine, you can know the maintenance and repairs required by the machine.

Dedicated helmet displays (HMDs) or heads-up displays (HUDs) are typical augmented reality devices, such as head-mounted devices (Microsoft's HoloLens), helmets or glasses, and handheld devices such as smartphones or tablets. Intel's Recon Jet Pro smart glasses integrate the functions of a smartphone into lightweight glasses, enabling interaction with remote enterprise resource planning (ERP) systems. The glasses are more than just a display module, allowing users to see the distant world through their eyes. Some vendors are expanding with other objects to generate augmented elements such as projectors, tables or desk lamps, or through the flat-panel display itself. With these improvements, AR users are no longer limited to bulky helmets or glasses.

As the aging population continues to grow, healthcare costs remain high in many areas. Advances in medical technology can not only reduce costs for hospitals, healthcare providers, and insurance companies, but also provide better, less invasive treatments. Here, augmented reality is used as a preventative measure, allowing medical professionals to receive data in non-traditional ways.

Cigna, a giant in the healthcare industry, just launched a project called "BioBall" this year. The project uses Microsoft's HoloLens technology to test patients' blood pressure and body mass index in the form of interactive games. Patients only need to hold a small ball and capture the images flashing on the screen within one minute. BioBall uses this to sense the patient's pulse and uses responsive light to connect the game to the patient's heartbeat. After the game, patients can listen to their health index in the headphones they wear, and the corresponding health advice will be sent to the relevant mailbox at the same time. Cigna pointed out that this technology is designed to encourage people to manage their bodies by understanding their own health data and enable people to receive targeted treatment.

The Augmentarium Virtual and Augmented Reality Laboratory at the University of Maryland uses AR to improve ultrasound work (Figure 1). Barbara Braun Cznanyi, deputy director of the University's Center for Health Informatics and Bioimaging (CHIB), pointed out that by using Microsoft's HoloLens and special software, doctors wearing AR devices can directly observe patients and ultrasound images in front of them, without having to observe them through a screen on the side.

Ultrasound is just the beginning for AR. “For surgical applications, we’ve only scratched the surface. We are actually planning to provide similar interfaces for other imaging modalities, while also conducting field tests to determine how well this tool actually works,” Braun Czanani said.

Many other companies are developing platforms using AR for more complex procedures. For example, Scopis is developing a tool that allows surgeons to augment their vision while performing spinal surgery.

At the same time, AR is opening up new directions for children's education. AR provides a variety of new teaching methods and stimulates children's interest in learning. In some cases, AR can also help children with learning difficulties catch up with other classmates. The AR project of the University of Helsinki in Finland helps children learn scientific knowledge by enabling them to interact with molecular movements in physics such as gas, gravity, sound waves and wind. They found that for these poorly performing children, AR is enough to help them narrow the gap with other students. At the same time, AR can also enable students who are already excellent in learning to go further and perform better.

AR also creates new types of learning by converting "old knowledge" into new formats. For example, the University of Helsinki said that AR makes learning more attractive because children have tried this technology when playing Pokémon Go. Therefore, using new technology to impart old knowledge is also the purpose of Shifu Orboot interactive STEM device. Orboot is the application of augmented reality in the education industry. It uses tablets or smartphones to enable children to learn about history, animals, monuments, language and art, weather and cultural knowledge through interactive 3D content.

The information is delivered through AR-generated 3D models, audio, and music, allowing learners to have a more immersive learning experience. Shifu noted that the aim is to promote active learning and memory storage, which is not possible through video content alone.

Projection-based augmented reality technology is becoming a new way to project virtual elements into the real world, and bulky helmets or glasses will gradually disappear from the stage of history. This is why AR is becoming a popular choice for home or office environments. Startups Lampix and Lightform are developing projection-based augmented reality technology for use in conference rooms, retail displays, hotels, digital signs and other fields.

Lightform’s device can be connected to any projector, similar to a Go Pro, and can instantly generate augmented reality elements (Figure 3). This will help multiple groups work together in an office, or create new elements in an artist’s studio, or showcase new products in retail signage applications.

"We think the entry point for this technology is five minutes," said Phil Renaris, design director at Lightform. "If it's a long period of time, wearing an AR or VR device makes sense. But if it's a faster experience - like shopping in a mall, you need a more seamless experience. At the same time, this technology can also allow many people to experience augmented reality at the same time without wearing expensive headsets."

Lampix has developed a lamp that uses a projection module, a visual module and a computer to achieve augmented reality. Lampix CTO and co-founder Miha Dumitescu pointed out that since there is no need for headphones or glasses, such devices can be used in conference rooms, so that collaborative work can be carried out both online and offline.

“AR can be seamlessly integrated into people’s daily lives. There is endless potential for applications that incorporate AR to improve people’s daily activities,” Dumitriscu said. “The focus should not be on the technology itself, but more on the experience. Lampix has great potential in achieving this goal. We don’t need to see the device physically, because it is just a meaningful experience.”

The work that companies and think tanks are doing around the world is what matters. In fact, augmented reality is trying to enter every industry and market, at least in the pilot stage. For example, Germany's FleetBoard is tracking truck logistics during the development stage of the APP to conduct a series of pre-inspections before departure. The FleetBoard Vehicle Lens APP (Figure 4) uses smartphones and software to provide real-time image recognition to determine the license plate of the truck. The relevant information is then superimposed in augmented reality, which speeds up the departure process.

FleetBoard is also working with Microsoft’s HoloLens to create a new way to manage truck fleets. Using an abstract road map, augmented reality allows fleet managers to monitor vehicles in a 3D environment to prevent accidents and delays, and alert drivers to potential problems on the road ahead.

Last winter, the Delft University of Technology in the Netherlands worked with emergency response teams to use AR as a tool for crime scene investigation. The handheld AR system allows on-site investigators to work with forensic teams in remote areas to solve cases to maximize the restoration of the scene. AR can also present multiple perspectives of the crime scene that are often overlooked by on-site crime solvers. This is very effective for tracking DNA traces, preserving evidence, and obtaining medical help from outside.

Sandia National Laboratories is investigating augmented reality as a tool to provide better security training for those who protect vulnerable areas, such as nuclear weapons and nuclear materials. Physical security training helps guide users through simulated real-life scenarios, such as theft or sabotage, so they can be better prepared when an incident occurs. This training can be completed using an AR headset, is affordable, and can be delivered anywhere.

VTT Technical Research Center of Finland recently developed an augmented reality tool for the European Space Agency (ESA) that allows astronauts to monitor equipment in real time in space. Space missions must be carried out at the exact time and without mistakes, and AR allows astronauts to conduct more in-depth practice by coordinating activities with experts in mixed reality. The tool visualizes telemetry data from space station equipment and other systems, such as diagnostic data and the latest maintenance data, life cycle, radiation, pressure or temperature, so that invisible elements can be traced - whether it is information from space or on the ground, it is displayed on AR glasses.

Daqri International in the United States uses computer vision in industrial AR to visualize data when working on a machine or in a warehouse. Daqri's glasses and headphones can display visual project data, tasks that need to be completed, potential problems with the machine, and even objects that need to be placed or repaired. The use of AR can improve the safety and effectiveness of industrial work spaces while making repairs, maintenance, and inventory management easier.

Augmented reality combines real-world objects or environments with virtual elements generated by sensory input devices, such as sound, video, graphics, or GPS data. AR operates in real time, interacting with objects found in the environment and overlaying virtual displays on the real environment. Although Pokémon Go is very popular among players, companies around the world have begun to use AR to solve real-world problems and needs. Law enforcement, education, logistics, construction and other industries are widely using AR technology and have achieved good results, such as more effective training, monitoring and collaboration. As for the future of AR? As AR technology continues to develop, a large number of applications using AR will rise like the dawn over the horizon.

All published articles are original and exclusive. Please indicate the source when reprinting. We reserve the right to pursue legal liability for unauthorized copying and non-compliant reprinting.