5G innovation: cars can communicate with others through virtual images, and even learn to drive from the "God of Cars"

A Japanese manufacturer wants to transform a car's windshield into a platform for hosting virtual engine images. At CES 2018, a Japanese automaker showed off what it calls "virtual to visual" (I2V) technology, and it's taking advantage of 5G to begin testing it in Japan's own test conditions. While the name sounds as flashy as magic, I2V is just a way to overlay information from the car's onboard sensors and the cloud onto the driver's field of view. Think of it as augmented reality (AR) on a larger scale, or a 3D heads-up display?

I2V relies on a system called Omni-Sensing as a hub for relevant data, which is organized and displayed as graphics floating around relevant objects - just like in a video game, you can see models of surrounding things made of 3D images. In fact, the entire platform was developed by Unity Technologies, which originally started in the gaming business. Drivers can also connect to what Nissan calls "Metaverse", a virtual environment shared with other users, just like you need a platform to play games, Unity Technologies provides a whole platform service. It allows people to appear in the car as virtual three-dimensional avatars, which is like the virtual images you see in the movies talking remotely.

Cars can also use this technology to warn drivers of impending hazards, such as poor visibility or road irregularities, or suggest alternative routes to avoid traffic jams. So, for example, I'm driving on this winding road, and the traffic coming from the other side is invisible to me, so I have to drive carefully because I don't know what's going to happen, which takes me longer than it should. I don't know where there's a pothole in the road. I don't know, around the corner, if the weather is going to be nice or not. So having all of this information presented to me as a virtual image is what I find most fascinating.

Nissan began testing the technology in Japan in March 2019, and explained that its real-world trials were made possible by ultra-fast 5G technology. The low-latency connection wirelessly transmitted the avatar data to the test vehicle, which is based on a van sold by the company in overseas markets, along with a live onboard camera view from the vehicle. The purpose of the trial was to evaluate how people riding in the car and those represented by the avatars perceive each other's presence through the user interface.

The technology is still years away from mass production. Even if it works in the real world, I2V seems distracting because you're always staring at a virtual image. The technology seems more suited to the self-driving cars of the future, which Japan Tech Research and other automakers are working to design and build.