Alipay medical model unveiled! Chinese and English exams surpass GPT4 and have been deployed in first-tier hospitals in Jiangsu, Zhejiang and Shanghai

Bai Jiao from WAIC
Quantum Bit | Public Account QbitAI

Alipay medical model unveiled!

Chinese and English medical examinations and benchmark tests reach or even exceed the level of GPT-4.

In addition, its medical professional capabilities ranked first on the A list and second on the B list in promptCBLUE, the Chinese medical LLM evaluation list.

The model has an accuracy rate of over 90% in identifying images of reports, medicines, hair, etc.

At WAIC, Alipay also jointly launched the AI ​​Medical Co-construction Plan with 20 institutions including People's Medical Publishing House, Peking University School of Medicine, and Zhejiang Provincial Health Commission.

Alipay Medical Model

Different from big models in other fields, medical big models pose higher challenges in terms of multimodality, security, and professional capabilities.

The first is multimodality, which is based on the Lark model.

At the beginning of this year, Ant Financial identified the technical direction of native multimodality and continued to invest in it. At present, the Bailing large model already has the native multimodal capabilities of seeing, listening, speaking, and drawing .

In other words, it can directly understand and train multimodal data such as audio, video, images, and text.

On this basis, it adds tens of billions of Chinese and English pictures and texts including reports, images, medicines, hundreds of billions of medical text corpora, and tens of millions of high-quality medical knowledge graphs.

At present, this multimodal medical big model can not only provide intelligent question and answer, medical record structuring and retrieval, and auxiliary diagnosis; it can also identify drugs and hundreds of medical reports, conduct hair health testing, etc.

These capabilities can be directly embedded in the entire medical process of hospitals and other institutions and quickly applied.

Secondly, in terms of professionalism and reliability, Gu Jinjie, head of Ant's big model application, revealed that Alipay took these measures during the R&D and model testing stages.

During the research and development stage, Alipay also cooperated with hundreds of professional medical teams and experts to label data to ensure that the information is authoritative and credible.

At the same time, we also collaborated with Shanghai Renji Hospital to launch the first Chinese medical specialty question-answering reasoning dataset RJUA-QA.

During the model testing phase, users, patients, and doctors participate in question-and-answer feedback.

Finally, in terms of security, in medical scenarios, it is crucial to ensure the reliability of technology and the privacy of data. Ant itself has been deeply involved in the security field for many years.

This time, Alipay also launched the "Medical Trusted All-in-One Machine + Trusted Cloud" solution for the industry.

The former realizes integrated training and push, and the coordinated optimization and acceleration of domestic computing power software and hardware solves the pain point of insufficient computing power in hospitals; the latter deploys dense reasoning through public and private clouds to solve data privacy and security issues.

Based on this large model capability, we have released full-modal digital humans, intelligent body-related products and industry solutions.

In fact, as early as last year, we had already started to deploy AI in medical scenarios.

In November last year, the Zhejiang Health Commission applied the "AI Medical Assistant" solution opened by Alipay and created the country's first digital health person "Anzhener" to provide Zhejiang residents with cloud accompanying medical consultations, health consultation and other services. Since its launch, it has served more than 1,000 medical institutions.

Some time ago, Shanghai First People's Hospital, using the AI ​​big model, not only launched Shanghai's first voice-interactive "digital accompanying doctor", but also pioneered the industry's generative electronic medical record, reducing the time required for cumbersome paperwork from 10 minutes to 15 seconds.

Earlier, Ant had already explored how to solve practical problems such as the difficulty of users seeing a doctor. In 2014, Alipay supported the first remote registration and payment in China.

After 10 years of development, the platform has served more than 600 million people seeking medical treatment. It is the largest third-party online medical insurance service platform and a one-stop medical and health service platform in China.

Currently, over 300 cities and over 3,600 hospitals across the country provide one-stop digital medical services on Alipay.

Ant's decade of accumulation in the medical ecosystem has also become the unique advantage and core barrier of their large-scale medical model.

It is no surprise that healthcare is one of Ant’s three core application areas.

Ant Alipay launches three major applications

Medical care, finance, life .

As big models enter the application era, Ant Group has also officially announced the three major areas in which they have landed, with the manifestation of professional intelligent agents :

AI medical assistant, AI financial assistant, Alipay smart assistant.

Among them, the Alipay Smart Assistant made its debut at WAIC and was selected as this year's treasure of the museum.

On Alipay, you can wake it up by simply pulling down from the homepage. Through conversation, you can complete services such as handling affairs, consulting, and taking taxis, and call up thousands of built-in mini-programs at any time.

For example, when ordering tea or coffee, Alipay currently supports the first batch of 12 tea brand mini programs including Starbucks, Luckin Coffee, Heytea, Bawang Chaji, and Mixue Bingcheng.

You just need to give the command, "I want to order a medium iced latte from Starbucks," and the AI ​​will enter the mini program to automatically place the order. After the user confirms and pays, he can pick up the coffee at a nearby offline store.

Ant Group Chairman and CEO Jing Xiandong revealed the reasons behind choosing the form of professional intelligent entity.

The industry generally believes that the implementation of general large models in rigorous industries faces three "capability shortcomings": relative lack of domain knowledge, difficulty in making complex decisions, and dialogue interaction does not equal effective collaboration .

In order to solve these problems, professional intelligent agents may be an effective path.

From our practice, professional intelligent agents are an effective way to implement large models in rigorous industries.

It is worth mentioning that the key technology supporting these three intelligent entities - intelligent cognitive decision-making technology and platform, also won the important award of this conference, "Excellent Artificial Intelligence Award (SAIL) TOP30".

This technology was jointly developed by Ant Group and Zhejiang University over a period of seven years. It has greatly improved the professional capabilities, accuracy and self-learning ability of artificial intelligence in serving rigorous industries, making AI and large models smarter and more reliable in making decisions.

So far, the project technology has been widely used in the medical, livelihood, finance, government and other industries, serving more than 2,600 hospitals and institutions such as Shanghai Renji Hospital, Shanghai First Hospital, Zhejiang Provincial Health Commission, and dozens of financial institutions such as PICC Health and Shanghai Pudong Development Bank, promoting the real large-scale implementation of artificial intelligence in the industry. The project has also been authorized with more than 100 invention patents, and some of its achievements have won the 2022 China Electronics Society Science and Technology Progress First Prize, the 2023 Wu Wenjun Artificial Intelligence Science and Technology Progress First Prize and many other awards.

Click here ???? Follow me, remember to mark the star~