The past of NEC Lab in Silicon Valley: The man who dragged Chinese companies into the AI era
Latest update time：2023-04-14
Reads：
"
The shining stars of NEC Lab in Silicon Valley are the prelude for Chinese Internet companies to enter the AI era.
"
Author | Li Mei
Editor|
Chen Caixian
On the eve of the Spring Festival in 2012, Robin Li took time to meet in Beijing with a technical young man who flew from Silicon Valley and was unknown in China. This man was in his thirties, wearing black-rimmed glasses, with a round face, and was not tall, but everything he said excited Robin Li.
This person was Yu Kai, then director of NEC Lab in Silicon Valley, a minority and devout believer in the early days of deep learning.
At present, he has a strong feeling that artificial intelligence will soon explode its technological potential in the industry, and the Internet in China is a fertile ground. This is a good time to return to China.
At this moment, Robin Li found Yu Kai. After Google completely withdrew from the Chinese mainland market, Baidu was very proud. Robin Li, who is sensitive to technology, closely followed the trends of top technology teams at home and abroad. He soon realized that artificial intelligence would bring new opportunities to Baidu, and began to look for talents everywhere.
When the two met, Yu Kai preached about deep learning, while Robin Li talked about his interest in speech recognition and image recognition. There is a kind of excitement and surging in the hearts of both parties.
It was on this day that Yu Kai and Robin Li made a historic mutual choice between Baidu and China AI: Robin Li extended an olive branch and invited Yu Kai to return to China. Driven by the tide of the times, Yu Kai resolutely ended his six-year career in Silicon Valley, returned to China and joined Baidu, established the multimedia department, and began to study speech recognition and image recognition.
Previously, Baidu's research accumulation in AI was almost zero. After Yu Kai joined, Baidu took the lead in injecting deep learning genes. In just a few months, Yu Kai sold Robin Li the potential of deep learning. Robin Li was shocked again and again, and even sent a letter to product managers across the company, asking them to pay attention to the technology of "deep learning". "Neural network", another synonym for deep learning, has also begun to spread in China. Baidu was the first to support the first batch of data annotation companies in China.
However, what really made Robin Li determined to invest heavily in deep learning was the failure of an auction in which Yu Kai participated on behalf of Baidu in December 2012.
The neural network AlexNet developed by Geoffrey Hinton's team won the world's top image recognition competition ImageNet in one fell swoop, causing a sensation around the world. As a former champion, Yu Kai was deeply touched, and he also knew the subversiveness of the AI technology paradigm represented by AlexNet. . So, he immediately sent an email to Hinton expressing his eagerness to cooperate.
This email inspired Hinton to launch a secret auction at the NIPS conference that year, attracting four companies from around the world that are now well-known leaders in the field of AI: Google, Microsoft, Baidu and DeepMind. Baidu bid tens of millions of dollars, but ultimately lost to Google. But through Yu Kai's eyes, Robin Li witnessed the madness of powerful rivals in the United States in pursuing deep learning, and felt the crisis.
In January of the following year, Robin Li prepared to establish the world's first AI laboratory named "Deep Learning" - Baidu Deep Learning Research Institute (IDL). Robin Li serves as the dean himself, and Yu Kai serves as the executive vice dean.
Yu Kai
Looking back many years later, that auction for the first time hinted at the competition between Chinese and American technology giants in the field of AI: After Google won, Hinton joined Google to start the wave of deep learning in the United States, while Yu Kai used Baidu's platform to gain talent in the field. The introduction and technological promotion have brought Chinese Internet companies into the next golden age like a singularity: the era of artificial intelligence.
Yu Kai was the early actual operator of Baidu IDL. Under his influence, Baidu attracted a large number of outstanding AI scientists, such as Xu Wei, the founder of Feipiao Platform, Huang Chang, later co-founder of Horizon, and Lin Yuanqing, later director of Baidu Research Institute. Coincidentally, historically they have planted their flags of visit at the same place: NEC Lab in Silicon Valley.
Focusing on this batch of talents that NEC injected into Baidu, especially Yu Kai, they also attracted more scientists, such as Ng Enda. Baidu's "Young Marshal Plan" was originally proposed by Robin Li to attract young talents. Yu Kai mainly lobbied Li Mu (later Amazon AI chief scientist and now founder of large model startup Boson.ai), Li Lei (later ByteDance Director of AI Lab) and Gu Jiawei (CEO of Xianwuling Technology) joined.
China's earliest autonomous driving project was also born in IDL. Autonomous driving veterans such as Ni Kai, Yang Wenli, and Zhang Tianlei were among the first batch of employees Yu Kai recruited for the project. Other figures such as Zhang Tong (later director of Tencent AI Lab) and Jia Yangqing (later vice president of technology at Alibaba and now an entrepreneur in the large model framework) have also interacted with NEC or people from NEC in the process of AI going to Chinese companies.
Most of them have become well-known AI scientists in Chinese companies, but the enclave that influenced them and the story of the enclave behind it are rarely known.
01
Talents from all walks of life gather in the "Heart of Silicon Valley"
In October 2009, the results of the International Vision Challenge Pascal VOC were announced at the ICCV conference in Kyoto, Japan. A collaborative team from UIUC and NEC won the classification track championship, with an image recognition accuracy 5% higher than the second place. NEC Lab in Silicon Valley began to become famous.
Pascal VOC was hailed as the "World Cup" in the computer vision world at the time. Gong Yihong, then director of NEC, gathered several young talents who were best at image research in the laboratory, such as Yu Kai, Xu Wei, Wang Jinjun, and Lu Fengjun, together with Yang Jianchao and Zhou Xi, students of Huang Xutao, the "Father of Chinese Vision" at UIUC, After investing five months to develop a new sparse coding model for image recognition, I never expected to win the World Cup in one fell swoop.
Gong Yihong was a little surprised. That year happened to be the tenth anniversary of his joining NEC Silicon Valley.
Ten years ago, in 1999, there was not a single Chinese scientist at NEC in Silicon Valley. He was the first Chinese to join NEC. Ten years later, NEC has gathered a group of outstanding young AI researchers. Although artificial intelligence was still tepid at the time, under their leadership, NEC America Research Institute gradually became one of the international centers of machine learning.
Gong Yihong
In the late 1980s, when the Japanese economy was at its peak, Nippon Electric Corporation (NEC) set up a research institute in the United States, divided into a Princeton headquarters and a Silicon Valley branch.
Gong Yihong was one of the first batch of Japanese government-sponsored students in China. He went to the University of Tokyo to study in 1983 and completed his bachelor's, master's and doctoral studies. After graduation, he briefly taught at Nanyang Technological University in Singapore and Carnegie Mellon University (CMU) in the United States, but he only worked for a few years. Finally, by chance, he came to the Media Analysis Department of NEC Lab in Silicon Valley and his career began. Steady takeoff.
Silicon Valley NEC Lab is a three-story building located in Cupertino (where Apple’s headquarters is located, known as the “Heart of Silicon Valley”)
Gong Yihong introduced the first batch of Chinese scientists engaged in artificial intelligence research to Silicon NEC Lab.
In 2001, he first found Xu Wei, a rare genius who had graduated from CMU with a master's degree.
Xu Wei studied as an undergraduate at Tsinghua University and began researching neural networks when they were still in their low ebb. After graduating in 1998, he went to CMU for a master's degree and proposed a neural network language model, which was the world's earliest pioneering study of natural language processing using neural networks (later "deep learning") methods.
According to Xu Wei's level, it was not a problem to continue his studies and obtain a doctorate. However, he wanted to apply neural network technology to engineering problems immediately, so he did not apply for a doctorate, but went out to find a job. Gong Yihong was very good at judging people and immediately introduced Xu Wei. Xu Wei is the first CMU talent recruited by Gong Yihong for NEC. At the same time as Xu Wei is "NEC Senior Sister" Han Mei.
Xu Wei
In the same year, Gong Yihong also recruited a summer intern named Zhu Shenghuo, who was in his third year of doctoral studies at the University of Rochester.
Zhu Shenghuo graduated from Zhejiang University with an undergraduate degree in 1994, and then went to Tsinghua University for graduate school, where he followed Huang Changning and Zhou Ming to do research on natural language processing and machine translation. In 1997, he went to the University of Rochester to further his studies in machine learning. In the middle of his Ph.D., Zhu Shenghuo came to NEC for an internship because of his outstanding mathematical ability. He passed Gong Yihong's interview and did multimedia research with Gong Yihong for several months.
When Dr. Zhu Shenghuo graduated in 2003, there were not many choices. At that time, the U.S. economy was in recession and domestic AI had not yet developed, so he first went to Amazon's machine learning team to develop recommendation algorithms. At that time, Amazon was still a small entrepreneurial team with only a few dozen people. However, after more than a year of contact with the business, Zhu Shenghuo felt that more in-depth research was still needed, so he decided to return to NEC.
Zhu Shenghuo
Not long after, Yu Kai, Wang Jinjun, Lin Yuanqing and others also arrived one after another. Among this, Zhu Shenghuo's "encouragement" played an important role.
In mid-August 2005, Zhu Shenghuo went to Salvador, Brazil, to attend ACM SIGIR, the top conference in the field of information retrieval. It was at this meeting that Zhu Shenghuo met Yu Kai for the first time.
At that time, Yu Kai was twenty-nine years old and had just graduated from the University of Munich with a Ph.D. Yu Kai is a native of Nanchang, Jiangxi Province. He completed his bachelor's and master's degrees at Nanjing University, then went to Germany to study for a doctoral degree. After graduation, he worked at Siemens Central Research Institute in Germany.
Zhu Shenghuo came to the conference this time. In addition to academic exchanges, he also planned to recruit some talents to join NEC. He had read Yu Kai's work before and was very impressed. It happened that he happened to meet Yu Kai this time, so he stepped forward to talk to Yu Kai. When talking about speculation, Zhu Shenghuo asked Yu Kai: "Are you interested in going to the United States?"
Yu Kai was planning to change his job, and he got along well with Zhu Shenghuo, so he responded very willingly.
Zhu Shenghuo treats people warmly, and Yu Kai is also very talkative. The two quickly became familiar with each other in Brazil. In the middle, Zhu Shenghuo encountered a robbery and was injured in a struggle. Yu Kai admired him and praised him for his courage. He smiled and waved his hands and said, "Where is it? When that person robbed me, I just kept following him." He said he could take it as he pleased, but people didn't understand. Yu Kai thought, "This guy is so silly. It's funny."
However, during that time, Yu Kai was also considering returning to China for development, and also received an offer from Microsoft Research Asia (MSRA). MSRA was led by Shen Xiangyang at that time and was the most famous AI research institute in the world. At that time, the NEC American Research Institute was also very strong. There were many great masters at the level of founders, such as Vladimir Vapnik, the inventor of SVM (Support Vector Machine), and Yann LeCun, the pioneer of deep learning. It was also very attractive to talents. At that time, if you searched for "NEC Labs American" on Google, a Google job advertisement would pop up with "private goods".
After much hesitation, Yu Kai finally decided to go to Silicon Valley for a tour first, based on the advice of his friends. Yu Kai, who entered NEC in 2006, was already determined that he would eventually return to China. No one expected that this idea would directly promote the transformation of the entire Chinese AI industry six years later.
Just like Zhu Shenghuo "motivated" Yu Kai at the academic conference, many subsequent people, such as Wang Jinjun, Lin Yuanqing, etc., also came to NEC through introductions and introductions at the conference. In those years, the scale of the conference was small, and people in the same circle could meet and greet each other at the conference.
Wang Jinjun arrived at NEC one month later than Yu Kai. At the 2004 ACM Multimedia Conference in New York, Wang Jinjun's doctoral supervisor Xu Changsheng introduced him to his friend Gong Yihong. At that time, Wang Jinjun was in his first year of doctoral studies at Nanyang Technological University and had already published a top conference paper on multimedia retrieval, which left an impression on Gong Yihong.
Two years later, Gong Yihong approached Xu Changsheng (now a researcher at the Institute of Automation, Chinese Academy of Sciences) who was working in Singapore at the time and asked if there were any good students he could recommend to NEC. Hearing Wang Jinjun's name, he immediately recalled, "I remember Wang Jinjun, it's good to have him here." After that, Wang Jinjun chatted with Gong Yihong on the phone across the ocean for more than an hour and passed the interview.
Wang Jinjun
Lin Yuanqing came to NEC in 2008 and was introduced by Yu Kai and Zhu Shenghuo. The three of them met for the first time in December 2007 at the NIPS conference in Vancouver.
It was the last year of Lin Yuanqing’s doctoral study at the University of Pennsylvania, and his paper was accepted as an oral presentation at NIPS. Previously, he studied for a master’s degree in optical engineering at Tsinghua University, and his doctorate focused on machine learning.
When Lin Yuanqing made a report, Yu Kai and Zhu Shenghuo noticed him. Both of them believed that his research level was very high, and that he was working in the direction of sparse learning (Sparse Learning) that was very relevant to the laboratory. They chatted with Lin Yuanqing about research. After hearing Lin Yuanqing say that he was looking for a job, they immediately told him that NEC was still recruiting people. Lin Yuanqing expressed his interest without any suspense and officially joined the job after graduation the following year.
During this period, NEC Lab in Silicon Valley was full of talented people. Most of them graduated from prestigious domestic universities, went to study abroad, and then gathered in Silicon Valley. There are many Tsinghua students among them. In addition to the above-mentioned Xu Wei, Zhu Shenghuo, Lin Yuanqing, etc., there are also Lu Fengjun and Yang Ming who came in in 2007 and 2008. Both of them graduated from Tsinghua University with a master's degree. Lu Fengjun is a programming master and has an undergraduate degree. He has published a book called "Introduction to Digital Image Programming and Processing", and Yang Ming is very good at computer vision.
According to many people's accounts, NEC is the place where they have the greatest freedom of research and is full of ingenuity. Researchers have little pressure on business delivery. For example, after the model is made, the business will be carefully reproduced. The original code may not be used, so everyone will joke, "Anyway, the things made will not be used in the product." NEC's funding source structure is also very special. The company provides 70% directly and only 30% needs to be obtained from BU, which allows them to focus more on research.
In this research environment, they quickly achieved outstanding research results, especially deep learning. Although the later explosion of deep learning was not at NEC, NEC was one of the first laboratories in the world to study convolutional neural networks (CNN).
In the 1990s, in China and the United States, one person insisted on sitting on the bench of neural networks - Xu Wei used neural networks in language models at Tsinghua University, and Yann LeCun studied ATM check recognition at Bell Labs.
Later, LeCun's team disbanded and went to NEC's Princeton headquarters, while Xu Wei was the representative of deep learning in the Silicon Valley branch. Xu Wei is very hands-on and wrote the laboratory's first convolutional neural network. Later, the methods of configuring model parameters such as Caffe and PyTorch were already implemented in the CNN implemented by Xu Wei.
After Yu Kai arrived, he also became the main force in deep learning in the laboratory. He gratefully believes that getting to know Xu Wei was his "biggest gain" from entering NEC, and it was Xu Wei who led him into the field of deep learning.
Xu Wei
Anyone who has come into contact with Xu Wei will emphasize his extreme intelligence. He has a keen intuition about problems and is extremely efficient. An example is that the bathroom on the third floor of the laboratory is relatively small. Before going out to eat together at noon every day, Xu Wei would always go to the bathroom on the second floor in order to avoid waiting for a few minutes. Everyone was surprised to find out that Xu Wei also adheres to his principle of efficiency optimization in such small matters.
At the same time, Xu Wei is very low-key and hides a lot of merit and fame. For example, in the Million Dollar Recommendation Algorithm Competition held by Netflix, a team named "xyz" ranked first for a long time, but never revealed their identity, which caused discussion at the time. This mysterious team is actually Xu Wei himself. Later, whether at Baidu or Horizon, Xu Wei also maintained the purest passion for AI.
With the efforts of Yu Kai, Xu Wei and others, NEC has achieved top-notch research in sparse coding, as evidenced by its 2009 Pascal VOC championship. It was also after this championship that the direction of their group began to diverge for the first time.
Silicon Valley investors came to the door and encouraged NEC and others to industrialize.
Gong Yihong is very excited. He has always valued the implementation of research and has long had the idea of starting a business. Coupled with the fierce financial crisis, the company was affected, there were constant rumors outside, and everyone was generally uneasy. During that time, even Huang Xutao, who had a high reputation in academia, was worried about not being able to raise enough funds to train doctoral students, so he had no choice but to urge students to graduate as early as possible.
After careful consideration, Gong Yihong asked everyone in private for their opinions, and it happened that Wang Jinjun also wanted to start a business. The two quickly established a company in California called AkiiRA Media Systems to build the next generation video search engine. However, bad luck meant that their industrialization attempt was not successful. The market environment did not improve, and they left within a year.
After that, Gong Yihong returned to China and served as chief scientist at NEC China Research Institute. In 2012, he went to Xi'an Jiaotong University to teach. Wang Jinjun first went to Epson Research Institute in the United States, and then returned to China to join Xi'an Jiaotong University in 2013.
02
Yu Kai creates a golden age
After Gong Yihong left, NEC found Yu Kai, who was quite popular in the team, and asked him if he was looking for someone to be the head of the laboratory, who would be more suitable?
Yu Kai blurted out "Xu Wei". He was also very anxious at the time, feeling that the company might not be able to survive, so he went out to look for a job and got an offer from Yahoo.
The company asked Xu Wei for his opinion again, and Xu Wei replied "Yu Kai" without hesitation. It can be seen that the two have enough trust in each other and both want to choose a good leader for the team before leaving.
Yu Kai has a strong leadership and often organizes gatherings at his home. People even suspect that all Chinese scientists in Silicon Valley gather at his home every weekend. Everyone is not very interested in leading a team, and is more willing to immerse themselves in research. It would be best if someone is willing to do the chores. Later, Yu Kai said with a "helpless" smile, "These guys are very clever."
However, Xu Wei left and went to Facebook before Yu Kai. Before he resigned, Yu Kai gathered everyone together and took a video. In the video, everyone was doing funny performances and saying goodbye. They have a good relationship with each other. They drive out to eat together at noon. After paying the bill, they will use the remaining change to buy a lottery ticket. They imagine that if they win the jackpot, they will not have to go to work next week. They went out to play on the weekend, and their chat topics ranged from all over the world at the beginning to family and parenting later, and they got to know each other deeply. So the moment of separation is not without sentimentality.
Looking back, the wave of talent exodus in 2009 was more due to external economic factors than the popularity of AI. But before the outbreak of deep learning, the group of people who stayed in this small building still contributed to NEC's final glory, and also made "unintentional" preparations for the AI boom in China's industrial sector after 2012.
Among them, Yu Kai was the turning point that led NEC out of the "cold winter".
At the end of 2009, Yu Kai decided to stay. He felt that he had the responsibility to reunite NEC's strength. He set two goals, "to prove to the industry that we can attract the best talents and produce the best results in the world."
On the first day he took over the laboratory, Yu Kai proposed that everyone should stop speaking Chinese in the office and recruit talents more openly. With this remark, everyone followed suit. The laboratory has mostly recruited Chinese people in recent years, and the headquarters has complained about the lack of diversity and asked them to recruit more diverse people. But there is no way, those white people and Indians often fail to pass the interview. NEC's interviews have always been strict. There are few people in the laboratory, so it is better to be lacking than to overdo it.
This time Yu Kai conducted interviews in person and recruited two great talents. One is Huang Chang, who graduated from Tsinghua University with a bachelor's degree and a Ph.D. He was very passionate about image recognition when he was an undergraduate, but until he graduated with a Ph.D., he had not yet figured out how to apply image recognition on a large scale in industry, so he went to the University of Southern California as a postdoctoral fellow. It wasn't until 2010 that Huang Chang rekindled his interest in the industry and found Yu Kai after being recommended by a friend. In Yu Kai's eyes, Huang Chang "is a genius just like Xu Wei" and his research in the field of face recognition is world-class.
The other is Indian Manmohan Chandraker, who has won multiple best paper awards. Yu Kai commented that he is "the best one in 3D vision." Later, he took over Lin Yuanqing's class, and now he is still leading the computer vision department at NEC.
As the head of the family, Yu Kai created good research conditions for the laboratory. He takes on most of the chores, often travels and runs projects, maintains a very good relationship between the laboratory and the headquarters, and has ample funds. Yu Kai has also established extensive cooperation with academia and industry, and is very active in external exchanges. For example, he has collaborated on papers with academic researchers such as Andrew Ng, and was invited to teach at Stanford University.
Yu Kai and Ng Enda met in Montreal, Canada in June 2009. In order to revive neural network research, Yu Kai and the later three giants of deep learning Yann LeCun, Geoffrey Hinton and Yoshua Bengio organized a deep learning seminar at the International Machine Learning Conference ICML and invited Andrew Ng, then a professor at Stanford University. give a report.
At that time, Yu Kai remembered Ng Enda, a tall man of the same age as him who spoke softly and elegantly, and the two maintained a good friendship from then on.
Interestingly, also in the winter of 2009, Yu Kai went from Vancouver to the Whistler NIPS branch venue and shared an off-road vehicle with several researchers to go up the mountain. The person driving the car at the time was Deng Li, another Chinese deep learning leader. At that time, Deng Li was using neural networks to launch a speech recognition revolution at Microsoft. Yu Kai didn’t know yet that he would soon set off a storm of deep learning industry change in China.
On the eve of the storm, it was the last golden age of NEC Lab in Silicon Valley.
In 2010, Yu Kai and Huang Xutao collaborated again, bringing together Lin Yuanqing, Zhu Shenghuo, Yang Ming, Lu Fengjun, UIUC interns Cao Liangliang and Zhou Xi, and Rutgers University professor Zhang Tong to win the first ImageNet Challenge Tournament champion.
Later, Yu Kai commented that the first beautiful thing he did after entering the industry was winning this competition, "It can be said that before AlexNet in 2012, we used unsupervised convolutional deep learning algorithms for image recognition. , is the best algorithm that has been widely proven.
The laboratory's reputation has improved, and everyone's previous pessimism has been wiped away. NEC continues to attract outstanding young people to join as internships.
In 2011, Yu Kai received an email from Professor Trevor Darrell at Berkeley, saying that he had a particularly smart student who wanted to do deep learning, but there was no project available at Berkeley, and he wanted to recommend him to NEC for an internship.
This student is Jia Yangqing. Yu Kai felt that Jia Yangqing was very qualified, so he immediately asked Huang Chang to guide him. It was here that Jia Yangqing came into contact with deep learning. The strong mathematical skills and engineering coding abilities he displayed during that time left no one doubting that this young man would soon come up with amazing things in the field of deep learning. Come.
Jia Yangqing
Sure enough, a few years later, Jia Yangqing made Caffe, the deep learning framework that influenced the world. Later, he also participated in the development of frameworks such as Tensorflow and PyTorch at Google and Facebook, which had a profound impact. After joining Alibaba in 2019, Jia Yangqing led the big data AI business and achieved industry leadership. Four years later, he chose to "graduate" from Alibaba and returned to start a business in the direction of AI architecture, which he was best at.
03
Lin Yuanqing follows the wave of deep learning
Compared with Gong Yihong, who became the "cannon fodder" of the times, Yu Kai's escape was more fortunate. In 2012, he returned to China at the best time and brought NEC talents to Baidu.
Before leaving, Yu Kai strongly recommended Lin Yuanqing to be the laboratory director. Lin Yuanqing has a strong sense of closed loop and does things from the beginning to the end. Yu Kai appreciates this very much in him.
Lin Yuanqing
During Lin Yuanqing's tenure, he continued to attract some talents to NEC. Among them are two alumni of the University of Science and Technology of China, Wang Xiaoyu and Yang Tianbao, as well as CMU graduate Zhou Feng. Well-known interns during this period include Xie Saining (now an assistant professor at New York University) supervised by Yang Tianbao. Later, he and He Kaiming proposed the famous ResNeXt.
When Yu Kai and his team were vigorously engaged in deep learning at Baidu, Lin Yuanqing, who was in Silicon Valley, also led the team to quickly invest in deep learning. Lin Yuanqing specializes in machine learning, so he responded the fastest and began to use deep learning to do large-scale fine-grained image recognition and 3D visual perception of autonomous driving.
They were the first in the world to use deep learning for object detection, and relied on their technology to win the runner-up in the 2013 ImageNet competition. Team members include Lin Yuanqing, Wang Xiaoyu, Zhu Shenghuo, Yang Ming, Yang Tianbao, and Han Xu from the University of Missouri. In the framework they use, deep learning is only used as a feature extraction tool. Although it ranks second, its technical originality is very strong. At that time, the Berkeley team was using deep learning for detection at about the same time as them, so after the game, Lin Yuanqing and Wang Xiaoyu were invited to Berkeley to report their new findings, and the audience responded enthusiastically.
Wang Xiaoyu said that it was precisely because they valued technological innovation and pursued perfection that after 13 years, the competition became more and more engineering-based and they stopped participating.
The team ranked one behind NEC in this competition in 2013 was Yann LeCun's team. There were a lot of exchanges between the two parties at that time. The code legacy that LeCun left at NEC was also passed down by the younger generations. Once, when Wang Xiaoyu participated in a face analysis competition, he used the code written by Xu Wei, and Xu Wei’s code Originally created by LeCun.
During this period, under the leadership of Lin Yuanqing, the papers published by the NEC team at the three top computer vision conferences increased several times, and they also won the 2014 CVPR Best Paper Award.
At the same time, an obvious change is that everyone has an increasingly strong desire to commercialize R&D results. But they are also frustrated. In an environment where AI is ushering in a boom in industrialization, there is not much room for technology implementation at NEC. NEC has always been a technology aristocrat, which allowed it to be at the forefront of technology in the past, but now, "Snow in the Sky" can only bring about disconnection from the market.
Therefore, around 2015, with the rise of AI in China, it became inevitable for NEC personnel to leave and return to China in large numbers.
04
Those who come from UIUC
Gong Yihong, Yu Kai, and Lin Yuanqing, these three leaders have built NEC Lab in Silicon Valley into one of the strongest artificial intelligence laboratories. As a result, this place has gradually become a must-go place for Chinese people, especially young students, to do internships.
At that time, doctors from various prestigious universities in the United States came to NEC for internships, such as the aforementioned Jia Yangqing, Xie Saining, etc. However, the one who provided the most young talents to NEC was Huang Xutao from UIUC.
In 2006, Gong Yihong visited Huang Xutao's group, and the two parties established stable cooperation. NEC provides research funding to Huang Xutao, who sends 1 to 2 outstanding students from his group to do internships every year. Research funds in universities are limited, and most students have a hard time in school. However, the treatment and environment at NEC are particularly good, so they are very willing to come. Every summer, they go to NEC to conduct research, start writing papers in September and October, submit papers to the CVPR conference in November, win the paper the next year, and attend conferences in March and April. This is such a process.
UIUC's computer science department is very strong and it is the only university in the United States that can rank with the "Big Four". As a disciple of Huang Xutao, the master of computer vision, his students are also the proud ones. Ning Huazhong, Yang Jianchao, Zhou Xi, Cao Liangliang and others have all interned at NEC and are also alumni of the University of Science and Technology of China.
Ning Huazhong graduated from the Class of 1996 of the University of Science and Technology of China. His master's degree continued in the Master of Science and Technology of China from Academician Tan Tieniu. He has gone to NEC for the most internships, for three consecutive years. Everyone joked that Ning Huazhong was sold to NEC. Yang Jianchao described that when Ning Huazhong went to intern, it was equivalent to a juncture between the group and NEC.
Ning Huazhong
Yang Jianchao came from Class 0206 of the University of Science and Technology of China and had the best grades in the class at that time. In 2006, he went to UIUC to study for a Ph.D., and collaborated with Yan Shuicheng (now the visiting chief scientist of Zhiyuan Research Institute), who was a postdoctoral fellow in the same year, and published his first influential article. Yan Shuicheng praised his algorithmic capabilities very much.
He was the intern who worked with Yu Kai the longest and collaborated the most. He went there twice in 2008 and 2009, doing research on sparse coding that was hot at the time. Ma Yi, who was teaching at UIUC at that time, was the first master to use sparse coding in vision. Yang Jianchao was his important inheritor. Their collaborative articles in this direction have become classics.
Yang Jianchao
In 2009, Zhou Xi was the main player in the Pascal VOC competition together with Yang Jianchao. Zhou Xi graduated from Class 9923 of the University of Science and Technology of China. His master's degree is also doing pattern recognition research at the University of Science and Technology of China. I went to UIUC in 2006, switched from speech to images, and won six World Intelligent Recognition Competition championships. Later, when he collaborated with Yan Shuicheng, Zhou Xi spent less than three days to implement the project algorithm that Yan Shuicheng had previously spent a month on, which made Yan Shuicheng sigh.
Zhou Xi
There is also Cao Liangliang, who graduated from Class 9806 of the University of Science and Technology of China. His master's degree went to Tang Xiaoou's Multimedia Laboratory at the Chinese University of Hong Kong. In 2010, he came to NEC for an internship. Many people have the impression that he is not only smart, but also "the handsomest guy among this group of people."
Cao Liangliang
Some of NEC’s regular employees are students of Huang Xutao’s students, such as Yang Ming. Everyone would joke that although the students who came to intern from UIUC at the same time as him were younger than him, in terms of seniority, Yang Ming had to call them "Uncle".
Wang Xiaoyu, who joined NEC in 2012, is similar. His mentor Han Xu is Huang Xutao's student. Two years before joining the company, Wang Xiaoyu went to NEC for two internships, doing large-scale image search research under the guidance of Yang Ming. He and Yang Jianchao were classmates at the University of Science and Technology of China, and later went to the University of Missouri to study for a Ph.D., becoming Han Xu's first Ph.D. Three months after Wang Xiaoyu enrolled, he collaborated with Han Xu and Yan Shuicheng to publish a one-author ICCV (Oral) paper, which became a stepping stone for him to enter NEC as an intern.
Wang Xiaoyu
These interns were a splendid array. Later, Yu Kai recalled them all as treasures and said with great pride, "We guided the growth of these shining stars to a certain extent in their early stages of growth."
At that time, the Chinese stars at NEC Lab in Silicon Valley were bright, and the research atmosphere was very free. Yang Jianchao still misses his time in Silicon Valley. "It is just a Research Lab. It is a very flexible and free team. No one particularly values product output."
This group of people's pursuit from 0 to 1 was something Wang Xiaoyu deeply felt when he first went for internship. When he first arrived at NEC, he was confused because he wanted to go in a new direction that had never been done before and no one at NEC had done it before, large-scale image search. He found that everyone will discover and solve new problems, rather than just producing papers in that direction according to which technology is hot. Wang Xiaoyu is very good at doing things to the extreme in a short period of time, so he gets results quickly.
Young talents who received training at NEC later went on to achieve careers in the AI industry and became well-known figures in various fields.
After Ning Huazhong graduated from UIUC in 2008, he first stayed at Microsoft for a year, then jumped to Gong Yihong's startup company at the time, and left in less than a year. He has stayed in the Bay Area since then, working at Google and Facebook. In 2019, he joined his senior brother Han Xu’s company, WeRide Zhixing, and now serves as the executive director of the perception team.
Yang Jianchao graduated in 2011 and spent several years at Adobe and Snap Research Institute respectively. Finally, he went to Byte and served as the head of the Silicon Valley AI Lab.
Zhou Xi finally chose to start a business. When the Chongqing Research Institute of the Chinese Academy of Sciences was being established in 2011, he was invited by Dean Yuan Jiahu to visit the thatched cottage in the United States and decided to return to China to contribute and attract talents to the country. Four years later, he further implemented facial recognition technology and founded Yuncong Technology.
Cao Liangliang worked in academia and industry many times, founded a company, and finally returned to the city where NEC is located, serving as a chief scientist at Apple.
05
Return and explore China’s AI territory
Let’s go back to the story of Yu Kai and Baidu.
After the establishment of IDL, Yu Kai began to look for top talents around the world for Baidu. He first introduced Robin Li to his old friends from NEC.
One day at noon, Yu Kai told Robin Li at the dinner table that if we want to engage in deep learning, we must recruit the best talents in the world. When Robin Li asked who he wanted to recruit, Yu Kai said Xu Wei's name without hesitation, just like he did four years ago.
Robin Li then asked how to attract Xu Wei, and Yu Kai said half-jokingly, "You have to treat him to dinner."
Unexpectedly, Robin Li arranged the meal right away. Yu Kai "fooled" Xu Wei back to Beijing from the United States and took him to Baidu's reception room, and then Robin Li came in.
Speaking of the entry-level treatment provided by Robin Li to Xu Wei, Yu Kai was very envious. When Robin Li invited him to Baidu, he didn't even treat him to a meal. Later, when Xu Wei joined the company, Robin Li sent a letter to all employees, saying that Xu Wei's joining would promote Baidu's technology research and development and contribute to the progress of all mankind. This shows that Robin Li attaches great importance to Xu Wei.
After Xu Wei joined Baidu, he discussed work with Yu Kai and agreed to build a deep learning platform. In order to quickly promote the project, Yu Kai specially applied for a conference room, posted the door sign "Paddle Closed War Room", and immediately roped in four people to work with Xu Wei. They worked from morning to night and quickly produced the first version of PaddlePaddle, which was used in IDL and other departments. Xu Wei therefore became the founder and founder of Baidu Feipiao.
Under Yu Kai's influence, Huang Chang, who was at Baidu's US R&D Center at the time, also returned to China and was responsible for Baidu's architecture work. He also worked with Yu Kai to build the image recognition technology route for Baidu. Zhang Tong then left academia and joined IDL, and later established the Baidu Big Data Laboratory as the vice president of the institute.
In 2014, Robin Li launched Baidu's "Young Marshal Project", and Yu Kai personally recruited three young marshals: Li Mu, Li Lei, and Gu Jiawei.
Li Mu started doing deep learning under the leadership of Yu Kai. At that time, Yu Kai made an exception and created conditions for Li Mu to join the "Young Marshal Project" while studying for a Ph.D. at CMU to guide him in deep learning.
Later, Li Mu, Chen Tianqi and others developed the MXNet framework, which also had a source in Yu Kai. Yu Kai once advocated open source PaddlePaddle, but was collectively rejected by Baidu. He felt it was a pity and believed that even if the open source framework is not developed at Baidu, others must do it elsewhere. So Yu Kai made an appointment with Li Mu to a cafe in Tsinghua Science and Technology Park one weekend and told him that the open source deep learning platform was worth trying. Li Mu Shen agreed and started taking action after returning to CMU.
Not long ago, Li Mu and her mentor Alex Smola left their jobs to start a business, reportedly aiming to create the most popular large-scale model at the moment. In fact, Li Mu discussed large models with Yu Kai when he was at Baidu. At that time, Yu Kai believed that by combining Baidu's voice, image, video and other projects, a large model could be trained, similar to today's ChatGPT. It's a pity that it was not really done. First, the resource investment was too large, and second, Li Mu gave priority to building an open source platform.
(Preview: With the rise of deep learning, a group of pioneers such as Jia Yangqing, Li Mu, and Chen Tianqi, as well as domestic companies represented by Baidu, took the lead in researching deep learning frameworks. Leifeng.com will write another article "Chinese Pioneers of AI Framework" in the future. Please continue to pay attention and welcome to add WeChat Fiona190913 to communicate).
Li Lei is currently an assistant professor at the University of California, Santa Barbara. He also joined IDL to do deep learning at the strong suggestion of Yu Kai. Two years later, Li Lei called Yu Kai one day and said he wanted to work on Baidu News as a recommendation system. When Yu Kai received the call, he was busy outside raising funds for Horizon. He immediately stopped on the side of the road and spent two hours persuading Li Lei that he should go to a company where recommendation business was more valued, even a small company.
After the phone call ended, Yu Kai immediately recommended Li Lei to Zhang Yiming himself, and finally Li Lei became Toutiao scientist and director of Toutiao Laboratory. Later, Mr. Yu Kai lamented to others, "Look, I persuaded Li Lei to come over. Is this a good decision?" At that time, Zhang Yiming was hungry for talents and searched for machine learning talents all over the world. He even publicly asked who could recommend top machine learning talents to him. If you have talent, give a car as a gift. Although Yu Kai has never seen the shadow of a car so far, he thinks that this Bole is very happy.
The third young marshal is Gu Jiawei. When Yu Kai first found him, it took him more than half a year to convince him to leave MSRA and join IDL. He is mainly responsible for the architecture research and development of human-computer interaction, and has led projects such as Baidu smart glasses and smart bicycles.
While recruiting young marshals, Yu Kai also formed China's first autonomous driving corps in IDL. The first group of people he recruited, including Ni Kai, Yang Wenli, Zhang Tianlei, etc., are now all stars in the field of autonomous driving.
Later, autonomous driving established a separate business unit, with Wang Jin (now the founder of Zhongzhixing) as the general manager, and Yu Kai’s support behind the scenes. At that time, Wang Jin was disheartened because his department was merged into the big search, and he wanted to leave. Yu Kai, who was about to leave his job, also sympathized with him and casually said to him, "How about you take over the autonomous driving?" past". Later, Robin Li did deliver autonomous driving to Wang Jin.
In 2014, Baidu's big events also included Yu Kai's introduction of Ng Enda.
During that time, Yu Kai often traveled to the Bay Area to recruit talents, and Baidu also set up an outpost laboratory in Cupertino to attract North American talents. In February of this year, Yu Kai went to the United States for a business trip and stayed at the Palo Alto Sheraton Hotel not far from the laboratory.
That day, Ng Enda also happened to be in this hotel. The two hadn't seen each other for two years, so they chatted about their recent progress by the hotel swimming pool. After a long time, they were still not satisfied, so they made an appointment to talk at a restaurant that night. Ng Enda talked about the online education Coursera he worked on after leaving Google Brain, and Yu Kai vaguely felt that his career was not progressing smoothly. The position of Baidu's chief scientist has been vacant for more than a year. This is a good time to recruit Ng Enda.
Just like what Steve Jobs said back then, "Want to sell sugar water for the rest of your life, or do you want to change the world with me?" Yu Kai tried out to his friend in front of him, "Instead of engaging in online education here, you might as well go back to what really moves you, Come up with the AI career that is your passion.”
This sentence made the frustrated Ng suddenly open his heart. Seeing that Andrew Ng was a little moved, Yu Kai continued, "Baidu is willing to open its doors to attract you."
Everyone knows what happened next. Ng Enda met with Baidu senior vice presidents Wang Jin and Robin Li, and officially joined the company in May.
Ng Enda during Baidu period
But what no one knows is that at least in April 2014, Yu Kai had intended to leave Baidu.
At that time, Yu Kai went on a business trip to Silicon Valley and met Ng Enda at a Starbucks in Palo Alto. Yu Kai told him some details about joining the company, including how to send out press releases. After saying this, Yu Kai changed the subject and told Andrew Ng in a sincere tone, "I plan to leave Baidu."
Ng was surprised and said, "You tricked me into coming to Baidu. You are the only good friend I have made on Baidu. I can't even speak Chinese, so don't run away."
In fact, at that time, Yu Kai had already had the idea of establishing a company like Horizon. He told Ng frankly that he wanted to do something in the chip field, and Ng also expressed his understanding, but hoped that Yu Kai would stay at Baidu for at least another year to help him transition. Yu Kai agreed and stayed at Baidu with his friends until he left on May 23, 2015.
Later, Robin Li tried to persuade Yu Kai to stay. After a long conversation, he suddenly asked Yu Kai as if he had a sudden realization, "Did you bring Andrew (Ng Enda) here because you wanted to leave and start a business?"
Yu Kai suddenly thought to himself that Robin's emotional intelligence is really high. He answered truthfully and explained that Baidu is the absolute number one in the field of AI in China and is closely following Google around the world. It must continue to attract the best talents and cannot interrupt this matter because of his departure. This is what he believes A promise from Baidu.
Yu Kai’s idea of making autonomous driving chips actually originated very early. When Yu Kai led Baidu IDL to use GPUs for deep learning on a large scale, he thought that the effect of using GPUs for deep learning was unintentional. Maybe there would be great potential in focusing on GPUs. Later, Baidu's deployment of autonomous driving verified his conclusion: without making chips, autonomous driving cannot be mass-produced.
In July 2015, Yu Kai established Horizon and recruited old acquaintances Huang Chang and Yang Ming as co-founders. The three of them have established a strong sense of trust and share the ideal of using technology to change the industry, so it is natural for them to come together again. Among them, Yang Ming came back from the United States. After leaving NEC, he went to the Facebook Artificial Intelligence Laboratory and proposed the famous face recognition algorithm DeepFace. However, Yang Ming later left Horizon to take care of his family in the United States, and now he serves as chief scientist at Xinyi Technology.
(From left) Huang Chang, Yu Kai, Yang Ming
Xu Wei also continued his third collaboration with Yu Kai on Horizon. Having followed deep learning for more than 20 years, he has a strong sense of mission for artificial intelligence. After leaving Baidu in July 2018, he decided to return to technical research and joined Horizon as the chief scientist of general AI.
Facts have proved that Yu Kai has bet on the industry. Unlike other Baidu self-driving veterans who chose to do full stack, he targeted processors from the beginning, and now he has ushered in his own era.
Zhu Shenghuo and Lin Yuanqing met again in Hangzhou in 2018. In his tenth year at NEC, Zhu Shenghuo received an invitation from Jin Rong to join iDST, Alibaba's new R&D center in Seattle. Later, when Bodhidharma Academy was established, he returned to China and went to Hangzhou.
Lin Yuanqing left NEC in November 2015 and joined Baidu at the invitation of Andrew Ng. He first served as the director of IDL and concurrently as the deputy general manager of the autonomous driving division. In 2017, he was promoted to the director of Baidu Research Institute. He also led the preparation of Baidu with Xu Wei on behalf of Baidu. National Engineering Laboratory for Deep Learning Technology and Applications.
Later, the structure of Baidu Research Institute was adjusted several times, and many unsatisfactory things happened. Lin Yuanqing has an easy-going personality, but he is still quite radical and principled in his work. He quickly chose to leave in September 2017.
Lin Yuanqing is a very persistent person. After finishing the ImageNet competition, he wanted to bring his research to the market. Later, he also wanted to do something more practical at Baidu. Now it was finally time to start a business, and Lin Yuanqing quickly founded Aibee in Beijing.
Some time ago, Lin Yuanqing met Zhu Shenghuo when he was doing camera business in scenic spots in Hangzhou. Zhu Shenghuo was also working on the offline implementation of machine learning at Alibaba. The two talked about this with great interest. Afterwards, Zhu Shenghuo went to Beijing on a business trip and chatted with Lin Yuanqing for a long time and decided to develop a career with him.
Now Zhu Shenghuo, as the co-founder of Aibee, is fully responsible for machine learning in Seattle. It is worth mentioning that Zhou Feng, who previously worked at NEC, also joined Aibee in 2018.
Wang Xiaoyu left NEC a little earlier than Lin Yuanqing. Instead of returning to China, he went to Los Angeles to create the Snap AI Research Institute with two alumni of the University of Science and Technology of China, Yang Jianchao and Li Jia (students of Li Feifei). Until 2017, he chose to return to China and serve as the chief scientist of Yuntian Lifei. As a star unicorn in the security industry, Yuntian Lifei has successfully listed on the Science and Technology Innovation Board in April this year. Not long ago, Wang Xiaoyu and Li Jia were elected IEEE Fellows together.
Wang Jinjun, who was the first to try industrialization, has always been enthusiastic about technology implementation after returning to academia. So in 2019, he founded Lingtu Technology to implement computer vision.
Up to now, the Chinese who came out of NEC Lab in Silicon Valley have taken root in the territory of China's AI and are growing wildly.
//
Recent popular articles
京公网安备 11010802033920号