A review of Toyota Intelligent Driving over the past ten years: Why did we give up on “end-to-end” after getting up so early?

Toyota Motor Corporation, as a leader in the global automotive industry, has been constantly exploring and experimenting in the field of intelligence . Our previous article also shared the direction of Toyota's intelligent exploration. Toyota proposed the currently popular end-to-end concept very early, and everything seems to be in place.

But the situation changed. James Kuffner, CEO of Toyota's intelligent company Woven, left last September; it was recently revealed that Toyota's intelligent driving solution in China will adopt Huawei 's MDC hardware + Momenta software solution.

What has happened to Toyota in the past ten years of intelligence?

From 2015 to 2023, we divide Toyota's journey to intelligence into four stages: establishment, new journey, integration, and adjustment.

Based on these four stages, this article will elaborate on Toyota’s important initiatives and strategic adjustments on the road to intelligence, as well as why it ultimately abandoned the “end-to-end” technology route.

Establishment period (2015-2019): initial exploration of research and technology

In the initial stage of intelligence, Toyota mainly strengthened the early research structure and expanded the development of vehicle active safety, autonomous driving technology, robotics technology, artificial intelligence , energy and materials. In 2016, Toyota established the Toyota Research Institute (TRI), focusing on the research and development of artificial intelligence, autonomous driving and robotics technology.

In 2017, TRI demonstrated the TRI P2, an intelligent driving test platform based on AI deep learning, and began to use Luminar's LiDAR for testing. In 2018, Toyota, Aisin and Denso jointly established the "Toyota Research Institute-Advanced Development" TRI-AD to accelerate the development of autonomous driving technology. At CES 2019, Toyota released the TRI-P4, an autonomous driving vehicle based on the Lexus LS500, demonstrating its latest achievements in autonomous driving technology. However, after that, TRI-AD did not release much new progress related to the intelligent driving vehicle platform.

By 2020, TRI-AD also formed a new holding company to continue developing intelligent technology. By the end of this stage, Toyota's intelligent driving technology is still in the early stage of forward-looking research and development and exploration, and no mass-produced models have been displayed or sold.

New Journey (2020): Establishing Organization and Business Expansion

In 2020, Toyota announced that TRI-AD will expand and strengthen its business by forming a new holding company, Woven Planet Holdings, Inc. (Woven Planet). At the same time, TRI founded Woven Capital to support the group's development by investing in growth-stage companies. At this stage, Toyota has built three maps in the field of intelligence:

Woven Core (development and implementation of autonomous driving technology) Woven Alpha (innovative business in connectivity, in-vehicle software and HD maps)

Woven Capital (venture capital in the field of intelligent driving technology)

It should be said that the technological exploration in the past few years has allowed Toyota to roughly sort out the intelligent technology know-how, which is concentrated in autonomous driving technology and network technology, and Toyota continues to make additional investments in these two areas. However, Toyota is also very clear that as the world's largest automobile manufacturer, early venture capital in this field can not only intervene in the embryonic stage of new technologies and lay out a networked ecosystem, but also help the parent company occupy a position in multiple technology tracks and spread risks. At the same time, on the electrification track, Toyota has extended an olive branch to BYD at this time . The two companies established a joint venture in China in March 2020, each holding 50% of the shares.

It is worth noting that the models jointly developed by the two parties at that time will carry the Toyota logo. This also laid the groundwork for Toyota's recent selection of Huawei as its intelligent driving hardware supplier, which has caused a sensation in the intelligent driving circle.

Integration period (2021-2022): Global shopping and technology integration

Entering 2021, Toyota has started a global shopping spree, and Woven Planet reports new developments on autonomous driving almost every month.

It acquired Lyft's intelligent driving team Level IV and invested in China's L4 echelon Pony.ai, Momenta and autonomous driving technology company Aurora.

In addition, it has invested in autonomous driving travel platform company Ridecell and acquired high-precision map company CARMERA. Woven Planet also acquired automotive operating system company Renovo Motors and invested in venture capital fund UP.Partners.

The vast majority of investments reflect Toyota's strategic investment attributes and are usually carried out after technical cooperation.

Faced with the rapid iteration of the autonomous driving industry, Toyota seems to have panicked. In 2021, Toyota released the Toyota Teammate Advanced Drive (TAD) new automatic driving assistance system , which is an L2-level automatic driving assistance system installed on its luxury brand Lexus LS model. This model was equipped with a forward laser radar and two side laser radars. The two side laser radars were Flash laser radars from Continental Automotive of Germany, namely HFL-110.

In 2022, Toyota and its Woven Core began to try to install the TAD (Teammate Advanced Drive) system on mass-produced models. The model is equipped with five millimeter-wave radars , four laser radars and eight cameras, covering the entire periphery of 360 degrees, and its luxury level is directly maximized. This function can support the driver to maintain lanes, adaptive cruise control, and automatic lane change and overtaking when necessary on highways, and is supported by high-precision maps, while also allowing the driver to drive hands-free.

The optional price of this system is US$5,170 (approximately RMB 33,000), and it comes with a 10-year autonomous driving software subscription service. However, Toyota may lack confidence in the promotion of smart driving, and only the top-of-the-line Mirai Limited and Lexus LS500h have optional packages for this function.

LiDAR needs to accurately detect the position and movement of vehicles in adjacent lanes to help avoid side collisions, and side/rear LiDAR needs to accurately detect free space near the vehicle in order to change lanes and merge into small spaces. In order to minimize its size, the space restrictions of the installation location also need to be maintained. To meet the requirements, the TAD system still uses Continental's Flash LiDAR, limits the detection range to a short distance, and eliminates mechanical components to reduce the size of the equipment.

At the same time, Toyota has not slowed down its exploration of L4 advanced intelligent driving technology. In April 2021, its subsidiary Woven Planet spent $550 million in cash to acquire Level IV, the autonomous driving car division of Lyft, a US shared travel service. It seems like a sky-high price now, but back to this time point in 2021, almost all global L4 solutions are financed in billions, and Toyota's acquisition of Level IV is not too extravagant. James Kuffner, who served as the team leader, was a member of the engineering team of Google's first self-driving car. Level IV impressed Toyota's intelligent driving technology solution, according to Level IV's own public introduction, including two stages.

First, in the Autonomy 1.0 stage, Level IV believes that the current intelligent driving development adopts the method of self-built intelligent driving test fleets, which basically runs the existing algorithm in the actual environment, verifies, modifies, and then verifies it again, and repeats the cycle of improvement. Looking at this Autonomy 1.0 stage from 2024, in fact, the intelligent driving solutions of most of the host manufacturers in the world are still in the 1.0 stage, and there is no practical change. The data is mainly collected from test vehicles. And obtaining data from mass-produced models, which is repeatedly promoted by everyone, is actually very difficult.

Many car companies use smart driving solutions from different suppliers for different models, and some are self-developed, which makes it difficult to achieve a closed data loop across vehicle platforms. Not only that, the repeated jumps on different technology paths also cause a lot of data waste.

From the early L4 solution reduction to achieve L2.999, to multi-source heterogeneous perception fusion, to the later BEV perception fusion, and then to the rise of pure visual occupancy network, in just two years, perception algorithms have experienced several peaks and troughs. On the contrary, those teams that firmly chose LiDAR + multi-vision fusion have made our first batch of "far ahead" intelligent driving models. At present, Huawei's small intelligent driving solutions all firmly choose the perception of LiDAR integrated in BEV perception, so as to achieve "end-to-end" performance that is no less than Tesla FSD.

Speaking of "end-to-end", we must quickly return to our topic, back to the Level IV Autonomy 2.0 stage. In fact, Toyota has been trying "end-to-end" development since 2021. Level IV defines Autonomy 2.0 for intelligent driving as a comprehensive AI machine learning system that covers all aspects of intelligent driving, such as perception, prediction, planning, and testing. The system uses a large data set containing rich real driving data for training and verification.