What is the ultimate smart device? Turn your car windshield into Google Glass!

    Imagine a car with a windshield that looks like a giant Google Glass lens. You no longer have to stare at the dusty road ahead or waste time watching the bad traffic on the street, because this huge screen will not only display vehicle and key road information, but also highlight tips related to driving safety, life convenience and entertainment information thanks to the deep integration with the vehicle data.

Application of Augmented Reality technology on car windshields

    OMG! Will the windshield display 3D images and data information? Although it seems a bit unrealistic now, Jules White, an assistant professor of electrical engineering and computer science at Vanderbilt University, said, "Although it will take at least a dozen years to realize the technological scenes presented in the movie "Minority Report", we can still see some good results in the short term."

    Professor White is a guest writer for the February issue of Proceedings of IEEE (Proceedings of the Institute of Electrical and Electronics Engineers). His research focuses on the application of augmented reality (AR) on automotive windshields. This issue of the journal includes eight papers on this topic, one of which is titled "Behind the Glass: Driver Challenges and Opportunities for AR Automotive Applications", which can give everyone a general understanding of this emerging field.

When glass meets the Internet

    Professor White gave a broad definition of AR technology: "Augmented reality" can be seen as the fusion of virtual information and real objects. The most typical application of "augmented reality" is the rear-view road condition device commonly used in cars. Through the connection between the camera and the dashboard monitor, the driver can clearly identify the traffic conditions behind the car.

    When the driver is reversing, he can not only see the dynamic image behind the rear bumper on the monitor, but also the distance to the front bumper of the next car after transposition processing, which is coded by different colors. In addition, the dynamic guidance function of the car will automatically determine the direction the car should go according to the position of the steering wheel, so that the driver can easily parallel the car.

    The head-up display technology currently used in some luxury cars has actually realized the application of "augmented reality" on the windshield on a small scale. Although only a small part can display information such as speed and turning direction, this is just the prototype of the application of AR technology on the car windshield.

    In the near future, the application of "augmented reality" on the windshield will be much smarter. For example, if a truck breaks down in the middle of the road ahead, or a drunk driver sways left and right, keeps changing lanes, etc., this information will appear directly on the windshield parallel to the driver's line of sight in a highlighted or bold form. In addition, when using the in-car navigation system, the driver does not need to stare at the display near the center console. He only needs to look straight ahead and look at the road ahead to get the corresponding road condition information.

    When driving at night, the on-board cameras and sensors combined with AR technology can enable drivers to more clearly identify road conditions, signs and pedestrians. Of course, with the further use of laser sensors and radars, perhaps one day drivers will be able to clearly understand the situation at turns at night.

Technology is a double-edged sword

    Compared to the well-designed user interfaces of consumer electronics and online products, the traditional practices of the automotive industry are somewhat outdated. Professor White said, "When designing a windshield like this, we will consider using the right combination of colors, lines and layouts to present the best user interface experience possible. Therefore, to achieve a breakthrough in the application of AR technology on the car windshield, we also face various challenges."

    One of the biggest concerns for researchers is whether this new technology can bring good technological effects. Professor White said, "Any technology that can help drivers make better judgments while driving should be used. But if it distracts drivers, that's another matter."

    The authors of the paper "Behind the Glass..." look forward to the emergence of a car with social functions in the future, the "Social Car". In their conception, every time the Social Car passes through an area in the city or encounters any passing citizen, their detailed information will appear in layers on the windshield of the car. This certainly sounds cool, but it can easily distract the driver.

AR windshield technology can support place name recognition

It's time to get off the line, buddy! Drive carefully!

    Other technical challenges to overcome include ensuring that drivers of different body shapes and heights can see the same image in different seat positions. There is absolutely no room for error, but whether to use a 3D display or ordinary flat-panel display technology is still undecided.

    Of course, we start with the most primitive and basic functions - navigation and speed control, followed by turning the turn signal, turning on the wipers, and then adjusting the air conditioning and entertainment equipment. To adapt to these different functions, different technologies and design methods must also be adjusted accordingly, which adds a lot of difficulty to the implementation of AR technology. Which functions can be realized by this huge windshield screen, and which functions must be controlled by the integrated instrument panel or steering wheel, etc. are all problems that AR technology needs to solve in the application of car windshields.

    Despite this, White and other researchers are optimistic about the application prospects of AR technology on car windshields, but due to current technical limitations, applications based on AR technology are still very simple. Professor White said, "A joint strike fighter is equipped with a helmet with high-tech technology worth about $250,000 for the pilot. Because as a combat fighter pilot, accurate data is crucial. But car drivers obviously don't need such complex windshield display technology."