Beihang University becomes the biggest winner of AAAI 2021, with two first-authors winning the best paper and nomination awards, all of which are related to Transformer
Yang Jing from Aofei Temple
Quantum Bit Report | Public Account QbitAI
On February 4th, local time, the AAAI 2021 Best Paper Award was announced!
Announced at the opening ceremony by Mausam, chair of the AAAI 2021 Program Committee, a total of three papers won the Best Paper Award.
And 3 papers were nominated for the Best Paper (Runners Up Award) .
It is worth noting that among these six award-winning papers, two of them have Beihang students as the first authors. From the titles alone, it can be seen that the research is related to Transformer.
△ Best Papers
△ Best Papers Runners Up
Beihang University becomes the biggest winner
First, the winner of the Best Paper Award is "Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting".
The first author is Haoyi Zhou from Beihang University, and the work was completed collaboratively by Beihang University, UC Berkeley, Rutgers University, and Beijing State Grid Fidelity Company.
This paper designs a Transformer-based model Informer designed specifically for LSTF (Long Sequence Time Series Forecasting) to solve some serious problems when Transformer is applied to LSTF.
For example, quadratic time complexity, high memory usage, and inherent limitations of the codec structure.
Informer具有三个显著特点:采用ProbSparse自关注机制、自注意力提炼、生成式解码器。
△ Informer model
In addition, two papers won the Best Paper Award.
One of the papers is from a Chinese team and is titled “Mitigating Political Bias in Language Models Through Reinforced Calibration”.
The first author is Ruibo Liu from the Department of Computer Science at Dartmouth College, whose main research is to mitigate the problem of political bias in language models by strengthening calibration.
And a paper from Singapore University of Technology and Design, called Exploration-Exploitation in Multi-Agent Learning: Catastrophe Theory Meets Game Theory.
A smooth simulation of Q-learning is studied to combine catastrophe theory with game theory to enhance the understanding of the tool of Exploration-Exploitation in multi-agent learning (MAL) .
Among the three papers that won the Runners Up Award , the first author of "Self-Attention Attribution: Interpreting Information interactions inside transformer" is Yaru Hao, who is also from Beihang University.
The second author is Dong Li from Microsoft Research Asia.
This paper proposes a self-attention attribution algorithm to explain the information interaction within the Transformer, and conducts extensive research using BERT as an example.
First, the most prominent dependencies in each layer are extracted and an attribution graph is constructed to reveal the hierarchical interactions within the Transformer.
Furthermore, self-attention attribution is applied to identify important attention heads, while other attention heads are pruned with only marginal degradation in performance.
Finally, the attribution results are used as an adversarial model to implement non-targeted attacks on BERT.
In addition, there is a paper titled "Learning From Extreme Bandit Feedback" from teams from the University of California, Berkeley, the University of Texas at Austin, and others, which studies the problem of batch learning from Bandit Feedback in an extremely large action space environment.
and, “Dual-Mandate Patrols: Multi-Armed Bandits for Green Security,” a paper by a team from Harvard University and Carnegie Mellon University.
The first author is Lily Xu from Harvard University, who explored how to use the Multi-armed Bandit algorithm based on reinforcement learning to prevent poachers from harming wildlife.
AAAI 2021 Opening Ceremony
This AAAI conference was held in the form of an online meeting, and the chairman of the conference was Yang Qiang, Chief Artificial Intelligence Officer of WeBank, Professor of Hong Kong University of Science and Technology, and leader of transfer learning .
Professor Qiang Yang is the second conference chair in the history of the AAAI conference and the first Chinese to hold this position.
A total of 1,696 papers were accepted for this conference, with an acceptance rate of nearly 21.4% , of which 70.6% were student papers.
The total number of papers from mainland China still topped the list, with 3,319 papers submitted, of which 627 were accepted, with an acceptance rate of 19.0%.
Chinese people performed well in various awards and won four major awards.
Including two papers from Beihang University, a total of four papers from Chinese teams won the Best Paper and Best Paper Runner-up awards.
There were also papers from teams from Shenzhen Key Laboratory of Computer Vision and Pattern Recognition, Southern University of Science and Technology, and other teams that won the Outstanding Paper Award.
In addition, two scholars from Microsoft Research Asia, Xiting Wang and Wu Fangzhao, received the Outstanding SPC Award.
The conference will last until February 9th. If you want to know more details, please click the link below~
Conference link:
https://aaai.org/Conferences/AAAI-21/
Best Paper/Runner-up link:
https://arxiv.org/pdf/2012.03083.pdf
https://arxiv.org/pdf/2009.12947.pdf
https://arxiv.org/pdf/2004.11207.pdf
https://arxiv.org/pdf/2012.07436.pdf
https://arxiv.org/abs/2009.06560
https://www.aaai.org/AAAI21Papers/AISI-8076.LiuR.pdf
-over-
This article is the original content of [Quantum位], a signed account of NetEase News•NetEase's special content incentive plan. Any unauthorized reproduction is prohibited without the account's authorization.
Join the AI community and expand your AI industry network
Quantum Bit QbitAI · Toutiao signed author
Tracking new trends in AI technology and products
One-click triple click "Share", "Like" and "Watching"
Advances in science and technology are being seen every day~