Dry information | Cases of cost reduction design errors: component selection and manufacturability design

Yun Zhongfa comes from Ao Fei Si
Qubit | Public account QbitAI

Tencent people don't talk much, and their large model technology just won two world championships!

Recently, WSDM (Web Search and Data Mining), the top international academic conference in the field of information retrieval, announced the results of the WSDM CUP 2023 competition.

The research team from Tencent won the championship in two tasks on the unbiased ranking learning and Internet search pre-training model tracks.

The ACM WSDM (Web Search and Data Mining) conference is one of the top conferences in the field of information retrieval. It is coordinated and organized by four special committees: SIGIR, SIGKDD, SIGMOD and SIGWEB. It enjoys a high academic reputation in the fields of Internet search and data mining.

WSDM Cup is organized by the WSDM conference. This year's WSDM Cup has more than 400 teams participating, from well-known universities and companies from China, the United States, Singapore, Japan, India and other countries.

There are three tracks in the competition:

• Unbiased Learning to Rank and Pre-training for Web Search (Unbiased Learning to Rank and Pre-training for Web Search);

• Multilingual Information Retrieval Across a Continuum of Languages;

• Visual Question Answering Challenge.

This time, Tencent's machine learning search team won the championship in two sub-tasks of the first track (Pre-training for Web Search and Unbiased Learning to Rank).

At present, both the code and papers of the two results have been published on GitHub.

Two task champion

In the field of deep learning, the quality of data annotation has a significant impact on the performance of the model.

However, the high cost of annotated data has always been one of the obstacles for the research team. How to technically use unlabeled data to train models has naturally become a hot topic in academia and industry.

In this competition, for the search-based pre-training task (Pre-training for Web Search), the Tencent team used methods such as large model training and user behavior feature denoising to conduct model pre-training based on search ranking on click logs, and then used The model is effectively applied to the retrieval task of downstream relevance ranking.

Through various optimizations such as pre-training, model fine-tuning, and integrated learning, it has achieved a large lead in the manually labeled relevance ranking task.

In another track - Unbiased Learning to Rank , the team dug deeply into click log information and made full use of features including document media type, document display height and the number of screen slides after clicking to classify documents. Correlation is estimated unbiased, and a multi-feature integration model that can integrate multiple bias factors is proposed, which effectively improves the effect of document ranking in search engines.

It is understood that the results of the winning team are based on the Tencent Hunyuan AI large model (hereinafter referred to as "HunYuan") and the Taichi machine learning platform .

At present, through the joint WeChat search team, the two technologies have been implemented in multiple scenarios of WeChat search, and have achieved significant effect improvements.

In April 2022, Tencent disclosed the development progress of the HunYuan large model for the first time——

HunYuan integrates CV, NLP, and multi-modal understanding capabilities, and has successively topped the list of five authoritative data sets such as MSR-VTT and MSVD, achieving a grand slam in the cross-modal field.

In May 2022, it reached the top of three internationally recognized CLUE lists at the same time, breaking three records in one fell swoop.

Now, HunYuan has made new progress, launching the country's first low-cost, implementable NLP trillion model, and once again reached the top of CLUE.

Tencent Taiji Machine Learning Platform is a high-performance machine learning platform that integrates model training and online reasoning. It has the training and reasoning capabilities of trillions of parameter models, and provides a complete end-to-end project for AI large model pre-training reasoning and application landing. Capability support, one-stop solution to algorithm engineers' engineering problems such as feature processing, model training, and model services in the AI ​​application process.

Tencent has long been committed to the research of cutting-edge search technology, improving the user search experience by improving search algorithms. The relevant technical team has rich practical experience in retrieval pre-training, large model training, search ranking task objective function design, etc., and has many research results. It has achieved leading results in international competitions and academic conferences for the first time, and is widely used in multiple business scenarios such as WeChat search, Tencent advertising, and games.

GitHub link:
https://github.com/lixsh6/tencent_wsdm_cup2023 Paper link: https://arxiv.org/pdf/2302.13756.pdf https://arxiv.org/pdf/2302.13498.pdf