ChatGPT thundered, and a storm suddenly erupted in the peaceful search world. Microsoft launched the ChatGPT version of the Bing search engine in a high-profile manner, forcing Google to rush to respond and launch the same chatbot based on the LaMDA model. As a result, Bard turned out to be buggy, and instead wiped off its own stock price by US$200 billion.
At the same time, Baidu also announced that its self-developed "Wen Xin Yi Yan" is accelerating internal testing and is expected to be released by the end of March.
The long-dormant technology giants have become nervous at the same time, not only due to the craze caused by ChatGPT among the public, but also because they have a hunch that the LLM (Large Language Model) application represented by ChatGPT is likely to be a "meta-technology".
"Meta-technology" refers to the fact that the continued use of the technology can in turn accelerate the progress of the technology itself.
For example: the application of chips in the real world can feed back R&D in terms of development efficiency, application scenarios and capital investment, thereby accelerating the progress of the chip itself; the development of "meta-technology" is in line with the law of exponential growth, so it can overcome common problems in R&D management. Explode complexity and escape the gravitational constraints of the “moderate progress trap.”
It is precisely because of the attributes of "meta-technology" that chip research and development can continue to achieve breakthroughs and catch up with the requirements of Moore's Law. In contrast, non-"meta-technology" such as biotechnology has to pick up the low-hanging fruits and then build Without a stepping stone for technological upgrading, it is stuck in the quagmire of Eroom's Law (anti-Moore's Law).
As everyone has experienced, using ChatGPT can greatly liberate work efficiency, including Open AI's own R&D personnel. Hundreds of millions of users have asked ChatGPT's fancy questions and repeatedly tested the "prompt black magic", which has also quickly accumulated a large amount of material for subsequent iterations of the version.
In addition, the huge cost of a single training of LLM and the occupation of computing resources also constitute a cruel entry barrier for new players.
As competition intensifies, both capital and computing power will rationally concentrate on technology leaders, thereby self-realizing Matthew's law that "the strong get stronger."
At this point, ChatGPT’s “meta-technology” flywheel has vaguely formed a closed loop. Therefore, if we do not get on board in time before the "technological singularity", the gap between the existing technology giants and OpenAI in AI applications will widen irreversibly, and eventually fall into an irreversible spiral decline; while those small AI entrepreneurial teams will face model generation. The gap cannot be made up by the gap and resources, and the disaster has arrived.
At a higher level, ChatGPT, as a "meta-technology", may have a nuclear bomb-level shock to China's future.
The progress of China's technology industry in the past ten years has relied on "fast follow" and "saturated execution" full of violent aesthetics in engineering. The basis behind this is the “engineer welfare” brought by the domestic education system.
Although we cannot cultivate S-level innovative talents, we can produce B-level standard products in large quantities: they are stable, diligent, strong in execution, and quite cheap, which can quickly fill the time gap from source innovation to application implementation. The overseas original creators were directly eliminated.
But as the best repeater in history, ChatGPT happens to overlap with the functions of "Chinese robots".
I can do what you know, and I can understand what you don’t understand. You can do 996, but I can do 007 365 days a year. Can you do it? What's more, ChatGPT is not a human being. He has no negative emotions to mess with his ears, no labor of daily necessities, will not resign, and will never lie down.
Being proficient in using ChatGPT is equivalent to hiring a team of assistants who are available on call, are durable, and have basic skills in various knowledge fields.
For S-level innovators who are over-the-top, ChatGPT provides round-the-clock support for the prototype implementation of their ideas; for innovative enterprises that gather S-level talents, ChatGPT means that implementation is no longer the main development bottleneck. . The downsizing of personnel and the improvement of intellectual concentration can further reduce the loss of internal information transmission, making it as agile and efficient as Open AI with only 400 people.
In this case, can the big manufacturer's strategy of using the open source framework fast follow still produce scale effects? Even if the cost advantage can be compressed, it will suffer from dimensionality reduction in terms of time efficiency. Some people in China often laugh at the fact that American universities, while producing a small number of S-level and A-level elites, produce a lot of garbage, so much so that they have to import B-level good employees from China and India for a long time; now they may no longer laugh.
The martial arts of riding and shooting that you were good at in the past suddenly became obsolete, and there is no place for you in the world of machine gun shooting. Therefore, if BAT does not grab tickets for this round of ChatGPT, the technology industry will miss this fatal "singularity moment." Regrettably, many domestic players are not only stretched out in terms of models, but are also about to face or even already face severe limitations in computing power and data.
LLM is greedy for computing power. A single training of GPT-3's 175 billion parameter (45.3T) model requires 1024 NVIDA A100 GPUs for 34 days. However, chips of the A100 level are currently embargoed. Next, the computing power required by GPT-4 may continue to increase by two orders of magnitude. What chips will we use to catch up at that time? After all, even 45nm process lithography machines are now embargoed.
The chip embargo’s suppression of domestic AI technology can be called a sophon’s lock.
Some people say that it doesn't matter. You can trade quantity for quality by stacking chips with mature processes, or you can eventually reach the same technical level by training slower.
The chip embargo is not to kill you with a hammer, but to make the implementation of any AI application financially unfeasible (or even thermodynamically unfeasible) by increasing the computing power costs of Chinese technology companies. You can only choose between losing profits or being forced to surrender, and you cannot enter a virtuous business cycle.
As long as the embargo continues and slows down the opponent's progress, the generation gap of "meta-technology" may not be converged after the "moment of singularity". If the results of the new round of LLM output are fed back into the chip development process, then this speed gap will further widen.
In terms of data sources, Microsoft and Google rely on the super-sovereign global Internet text. Taking GPT-3 as an example, the data mainly comes from Common Crawl, with a total of 3.1 billion pages, covering Internet text data from 2016 to 2021, and WebText2 is used as a high-quality document class to perform quality filtering on the original data; in addition, 2 are imported Additional additions have been made to the library and Wikipedia.
The Chinese Internet that BAT relies on is not large enough (less than 5% of Chinese data in Common Crawl), and the data ecology is not good enough.
The information credibility of most Chinese pages is not as good as Baidu Tieba. Very little high-quality data is trapped in the "walled gardens" of various manufacturers' private domains, and quite a few links result in 404 when opened. As for supplementing with structured databases, are you referring to CNKI and Baidu Encyclopedia?
Training with a data set lacking this kind of quality and structure is likely to result in a garbage-in-garbage-out nonsense generator, such as Baidu’s Plato robot that was hastily taken offline.
Under the current data sovereignty and information security regulations, it is also difficult to use overseas data to "digitize data from the west and calculate it from the east" and then translate it back into Chinese. Thinking more deeply, how to eliminate the implicit ideological implantation in the “imported model”? "According to relevant laws, this answer will not be displayed"? As shrewd as Microsoft, it blocked the permission of Chinese IPs to use the ChatGPT version of the Bing search engine from the very beginning, thus permanently ruling the country.
The fatal moment of singularity is sounding the alarm.
Add WeChat reply "
Join the group"
Invite you to join the technical exchange group!
Domestic core|Automotive electronics|Internet of Things|New energy|Power supply|Industrial|Embedded…..
Reply to any content you want to search in
the
official
, such as problem keywords, technical terms, bug codes, etc.,
and you can easily get relevant professional technical content feedback
. Go and try it!
If you want to see our articles often, you can go to
our homepage, click on the "three little dots" in the upper right corner of the screen, and click "Set as star".
Welcome to scan the QR code to follow
京公网安备 11010802033920号