China Unicom’s Zhang Yunyong professionally interprets chatGPT. What insights does it give us?

At the turn of the old and the new, the ghost of a universal program is wandering around the world. This ghost is ChatGPT developed by Open. It came out so quickly that it already had over 100 million users before it even had a Chinese name. ChatGPT, abbreviated as Chat Generave Pre-Trained Transformer, has quickly become popular in the Internet world. In just 2 months, it has achieved the small goal of breaking through 100 million global users. In order to achieve this goal, the phone took 75 years, 16 years, the website took 7 years, Apple's application software iTunes and AppStore took 6.5 years and 2 years respectively, and the social software Twitr, WhatsApp, Instag, and Douyin took 5 years, 3.5 years, 2.5 years, and 9 months respectively.

The reason why ChatGPT has been able to communicate with 100 million users in a short period of time is mainly due to its powerful interactive capabilities. It can not only create poems and songs, but also revise them, solve certain mathematical problems, and write professional academic papers. Especially for paper writing, it has caused a lot of controversy. According to some news reports, 89% of students in the United States use ChatGPT to write papers and complete homework, which has also prompted many universities around the world to ban students from using ChatGPT.

It can be seen that compared with some other chatbot applications, ChatGPT has indeed demonstrated extraordinary functional upgrades and performance improvements in terms of communication accuracy, breadth of knowledge, contextual understanding, multi-round dialogue interaction, user intention guessing, active admission of mistakes, and questioning of user questions. In some scenarios, interacting with ChatGPT is very similar to real interactions between people, learning from each other, sharing, and expressing opinions.

From the perspective of theoretical research and technological innovation, it can be determined that the technology used by ChatGPT is the brightest star in the current wave of generative artificial intelligence technology (AIGC, AI Generated Content). Generative artificial intelligence technology expands the application boundaries of decisive artificial intelligence, uses artificial intelligence technology to generate content, and breaks the "monopoly" of professional generated content (PGC) and user generated content (UGC) in writing, painting, music, education and other fields for creative work, just like using ChatGPT to create poetry and write manuscripts.

As a newcomer in the field of generative artificial technology, the core of ChatGPT is the GPT model (Generative Pre-trained Transformation Model), which is a natural language processing model. Its design idea is to learn language patterns by training the model in a large text corpus through specific prediction of the probability distribution of the next word, and to generate natural language text based on this. From the advent of the GPT-1 model in 2018 to the popularity of the GPT-3 model today, the intelligence of the GPT model has continued to improve. The larger corpus and parameter scale, higher accuracy and adaptability, stronger computing power and self-learning ability, and more general pre-training have made ChatGPT's functional upgrades and performance improvements.

The GPT-4 model is expected to be released in early 2023. Compared with GPT-3 and 3.5, the performance of GPT-4 will be greatly improved. Of course, the simplest technical principle of ChatGPT comes from probability statistics. The simple, elegant, profound and meaningful Bayesian theorem is the basis of its development. Through Bayesian theorem, ChatGPT can calculate the probability of generating a sentence in a known language pattern, as well as the probability of making a corresponding specific answer to known information and questions, so as to select the optimal response and realize interaction. This once again shows that mathematics is leading the human world forward, and the digital economy is also a mathematical economy.

In the digital age, the large-scale popularization of any application requires the dual drive of technology and market. ChatGPT is no exception, so we have to mention ChatGPT's R&D team OpenAI. In 2015, OpenAI was founded by well-known entrepreneurs Elon Musk, Altman, Peter Thiel and others. It is an artificial intelligence research company with financial support from many heavyweight companies in Silicon Valley.

Recently, Microsoft officially confirmed that it will continue to invest billions of dollars in OpenAI, and the subsequent investment is expected to reach 10 billion US dollars. This will surely become a good story in the field of artificial intelligence. OpenAI was a non-profit research institution in its early days, but now ChatGPT is no longer open source and has officially started commercial operations. It is not difficult to see that there is still a fierce game in the capital market behind ChatGPT.

Microsoft announced that it would apply ChatGPT to its search engine Bing. Google also invested heavily in natural language processing, developing the BERT model (Bidirectional Encoder Representation from Transforme) and planning to launch a chatbot named "Bud" to compete with ChatGPT. These facts further confirm that Internet giants are using the capital market to accelerate their open and covert struggles to define the heights of technological innovation.

△Photo source: Beijing Daily WeChat official account

ChatGPT has demonstrated powerful performance in terms of user experience. However, some people have begun to worry that individuals or industries may be replaced by artificial intelligence. Several founders of OpenAI have discussed related issues. Among them, Altman said: The purpose of developing generative artificial intelligence technology is to provide tools and expand capabilities for human creators. It is designed to enhance rather than replace them. All deep biological things cannot be replaced, including the motivation to interact with others, have fun, and create new things.

Some people also asked ChatGPT through experiments which occupations it would replace, and the answer was four aspects: data entry and processing, serving and helping customers with common problems, translation tasks, report writing, and content generation. However, ChatGPT also showed a full desire to survive and added: "It is important to note that while some tasks may be automated, technology like mine can also help and enhance human workers, making them more productive and efficient at work."

From another perspective, the problems and drawbacks of ChatGPT in terms of technology and cost also show that it cannot truly replace industries or individuals at present. ChatGPT lacks "human common sense" and the ability to extend in areas where it has not been trained with a large amount of corpus, resulting in serious "nonsense"; it cannot handle complex, lengthy or particularly professional language structures, such as medicine, natural sciences, etc.; and it lacks the ability to handle and accept new online knowledge.

At the same time, it is still a black box model and cannot ensure that no offensive expressions are generated. In order to make the content generated by ChatGPT more in line with human habits, OpenAI has hired more than 40 people to score the generated content. These 40 teachers only give high scores to helpful, true, and harmless texts, and low scores to those containing bad content. This also makes ChatGPT, this student, gentler and more full of positive energy.

ChatGPT also requires very strong computing power support. From GPT-1 to GPT-3, the number of model training parameters has increased from 117 million to 175 billion. The cost of training GPT-3 once is 4.6 million US dollars, and the total training cost is as high as 12 million US dollars. This figure is already extremely shocking, but according to a company's calculations, the cost of using ChatGPT for the company is about 100 billion yuan per year, which is jaw-dropping.

Based on the above analysis, it is not difficult to find that chatbots such as ChatGPT do not really have the ability to analyze, understand, and judge. What they can do is more like repeating what others say, or repeating what others say after weighting, which is closer to the real scene. It is an AIGC framework closer to upper-level applications. In essence, it still needs the support of underlying computing power. It is artificial weak intelligence that is calculated as intelligence. There are many evolutions worth looking forward to in the future, such as further improving the algorithm to make it more compatible with data and computing power, further strengthening the transformation of modalities based on the conversion between text and images, and improving the ability to continuously optimize and upgrade itself during use. Breaking through these problems will promote innovation to extend to a wider range of application scenarios, and also promote artificial intelligence to continue to move towards perceptual intelligence and cognitive intelligence, so as to better help humans create new knowledge and promote social progress.

Finally, whether we define ChatGPT as a milestone in the innovation of generative artificial intelligence technology, or reveal its dependence on human labor and the ruthless game in the capital market it has triggered, an indisputable fact is that the emergence of ChatGPT seems accidental, but it is inevitable. Since its establishment, OpenAI has gone through eight years of trials and tribulations. Its long-term persistence in its ideals has transformed into ChatGPT's achievements today. This is worthy of our deeper thinking about China's scientific and technological innovation.

In 2021, the number of patent applications in the field of artificial intelligence in China exceeded that of the United States for the first time, but there is still a big gap in key technologies, core algorithms, etc. For example, the deep residual (ResNet) algorithm behind ChatGPT was proposed by Microsoft, and the paper was the best paper published in the top conference "CVPR" (CVPR); another cornerstone algorithm, Transformer, was proposed by Google and uses a unique mechanism to process all input data at one time.