One year after the national standard was issued, why is the installation of CMS products in vehicles not up to expectations?

GB-15084 will officially take effect on July 1, 2023. After ECE R46 was promulgated and took effect in 2016, vehicles equipped with CMS systems (CMS is the abbreviation of Camera Monitor System, literally translated as "camera monitoring system") began to be sold in foreign countries in 2018. So far, more than a dozen models have been launched in the international market, while only two passenger cars have started mass production in China.

At this year's Beijing International Auto Show, there were not many vehicles equipped with CMS systems. So far, there are only a few passenger car models that have been announced for mass production. So, what is the reason why this product that everyone expected to be popularized quickly failed to meet expectations?

The right time, right place and right people are the key factors for the success of a thing. If we use this concept to explain the application of CMS system in vehicles, we may be able to get a glimpse of the clues. As a person who invested in the research and development of CMS system earlier in China, I have summarized the following views based on many years of research and development experience and market observation, and I am willing to discuss this once-in-a-century disruptive product with colleagues in the industry.

1. There are many companies that claim to develop CMS products, but few have the capacity to produce them in quantity . The reason is that the threshold for entering the CMS market is high.

With the launch of vehicles equipped with CMS products internationally, after the first CMS national standard seminar was held in Chengdu in 2019, automotive parts companies and OEMs began to pay attention to the CMS product.

The application of vehicle-mounted cameras has a history of more than ten years, starting with the incremental application of rear-view cameras in 2010, the incremental application of surround view cameras in 2017, and the incremental application to ADAS before 2020. After more than ten years of development, the vehicle-mounted camera technology has become relatively mature.

The number of in-vehicle display electronic products has increased since 2017, and LCD instruments have become very mature after several years of iteration. Their safety level is the same as that of CMS and both meet the requirements of ASIL B.

The hardware and software of electronic products are also very mature in vehicle applications. The three major components of CMS are mature products in the automotive electronics industry, leading many people to believe that only by optimizing the image color and field of view can the national standard test be met.

As a veteran who has been engaged in CMS research and development for 8 years, I initially held a similar view, but gradually changed my mind during the process of research and development, experimentation, and iteration, and became more and more in awe of this product.

CMS products integrate cross-industry knowledge such as optics, electronics, physics, and mathematics. Automobiles involve many parts and the industry is clearly segmented. Talents in these disciplines are rarely gathered in one company. Talent shortage is only a technical problem, which can be overcome through recruitment and resource integration, but patent breakthroughs require a lot of time and cost.

Intellectual property (patents) are divided into superior patents and subordinate patents. In order to protect their patents, companies in Europe and the United States will apply for patents as long as they have creative ideas. At this time, the patent is only a framework and basically does not involve the details of the implementation process. During the R&D implementation process, they will continue to apply for patents for implementation details. Whether it must be applied for or disclosed depends on the patent holder's observation and control of technology and the market.

Assuming that other natural persons or companies have similar ideas, they cannot apply for the disclosed patent content, but can apply for a detailed patent for the implementation method, which is a subordinate patent. The subordinate patent holder or other natural persons or companies can circumvent part or all of the rights protection of the disclosed implementation method patent on the basis of these two patents and apply for a completely new implementation method, which can also be a subordinate or iterative patent.

The lower-level patent is subject to the constraints of the upper-level patent, but if the upper-level patent holder mass-produces the product involving specific implementation details, it will also infringe the rights of the lower-level patent holder. In this case, in most cases, the negotiation is handled through patent cross-licensing.

If a natural person or company without superior and inferior patents wants to produce and sell products, they must first obtain authorization from the superior and inferior patent holders separately or simultaneously before they can engage in commercial production and sales.

Since the first CMS-like prototype was launched in 1987, more than 30 years have passed. During this period, countless people have participated in R&D, trial production and iteration, which led to the introduction of the ECE R46 regulation, making this seemingly conventional new combination a safe and reliable choice to replace the mass production of glass rearview mirror products. Their hard work has been transformed into intellectual property protection.

For example, regarding the display solution of the CMS system, many people question whether the field of view required by regulations can be applied for a patent?

Don't worry, the regulations require that there are three levels of vision, the most basic of which is like a mirror. The higher the level, the higher the flexibility, and the more the display area meets the driver's precise observation needs. The regulations do not specify what display solution to use, and companies can use different solutions to achieve it. Among them, there must be the best solution that best meets the needs of drivers. Companies that enter the market first and conduct extensive research and testing will have the opportunity to master this solution technology and form a patent.

The glass reflector is a whole, and its display area depends on the area and reflection angle, while the observation area of ​​the CMS display is also limited by the effective display area of ​​the display. Achieving a wider observation field of view within the limited effective display area to meet the driver's observation area changes under different vehicle postures, while also meeting the regulatory requirements for field of view, magnification, aspect ratio and other parameters is a technical problem that needs to be solved.

Interpreting the GB-15084 regulation is only the beginning of CMS product development. The regulatory requirements are converted into a data model through mathematical calculations combined with optical knowledge, electronic software and hardware, so that the display screen can be moved and changed without affecting the two regulatory test indicators of the magnification and aspect ratio of the observed object. The driver judges the distance and speed between the vehicle and the observed object based on the image size of the observed object on the display and the time of size change. The distance and speed of the object are crucial to the safety of driving a vehicle. It is not presumptuous for the technical implementation method of expanding the field of vision to be patented.

The implementation of GB-15084 is indeed the basis for CMS product development, but to transform regulatory requirements into practical technical solutions, it is necessary to combine data models with mathematical calculations, and then cooperate with optical knowledge and the application of electronic hardware and software. Such comprehensive applications enable the CMS display screen to not only move and change, but also maintain the magnification and aspect ratio of the observed object, meeting the requirements of regulatory test indicators.

By observing the size of the object imaged on the display and the time it changes, the driver can infer the distance and speed of the object from the vehicle. Therefore, the distance and speed of the object are crucial to the safety of driving the vehicle. The technical implementation methods for expanding the field of view are of great significance, and the innovative application of these methods can be patented.

The above two pictures show the changes in the display area of ​​the vehicle in different postures, which can be more appropriately interpreted by the term "human-computer interaction".

Cameras, displays, and PCBAs have formed a mature product combination that constitutes the CMS system that needs to be redesigned. The improved technical points almost subvert the original products and are only based on the basic principles of the original products.

For example, in terms of cameras, the design of the lens needs to be based on the glass mirror hemisphere design principle, combined with the vehicle's three-dimensional data, the normal of the human eye, and the default field of view observation area required by the GB-15084 regulations. By calculating the optical design parameters of the CMS lens and continuously adjusting and optimizing it using optical design software, a CMS lens that is similar to the glass reflector principle was finally designed.

The CMS camera lens should not be installed tilted toward the outside of the vehicle body, because doing so would violate the optical imaging principle, resulting in perspective distortion and changes in magnification and aspect ratio indicators. This inconsistency is seriously different from the imaging method of glass mirrors, which will have a negative impact on driving safety and cannot achieve the purpose of replacing glass mirrors.