He Kaiming and Chen Xinlei's latest research: Proposing a new instance segmentation method TensorMask, which is comparable to Mask R-CNN

Tong Ling from Aofei Temple
Produced by Quantum Bit | Public Account QbitAI

The team of great He Kaiming has new research!

This time, Facebook's Chen Xinlei, He Kaiming and others once again solved the difficult problem of instance segmentation tasks from a new perspective:

They proposed a general instance segmentation framework TensorMask , which makes up for the shortcomings of dense sliding window instance segmentation .

The test instance segmentation results on the COCO dataset show that the effect of TensorMask is comparable to Mask R-CNN.

What kind of new approach is this?

4D Tensor

In the paper TensorMask: A Foundation for Dense Object Segmentation, researchers introduced the general framework TensorMask in detail.

Previously, the mainstream method for instance segmentation was Mask R-CNN, in which the model first detects the bounding box of the object and then performs cropping and object segmentation.

However, Dense Sliding-window Instance Segmentation has received little attention. In this method, the output of each spatial location has its own geometric structure of spatial dimensions, which is fundamentally different from Mask R-CNN.

To formalize this method, the researchers regard dense instance segmentation as a prediction task on 4D tensors and propose a general framework TensorMask to capture this geometric structure.

The core change of TensorMask is to use structured high-dimensional tensors to represent the image content in a set of dense sliding windows.

TensorMask contains two parts: one is the Head for predicting the mask, which is responsible for generating the mask in the sliding window, and the other is the Head for classification, which is responsible for predicting the category of the target.

The two have clear division of labor and each performs its own duties.

Performing instance segmentation detection on the COCO dataset shows that the average accuracy of TensorMask on test-dev reaches 35.5, which is very close to 36.8 of Mask R-CNN.

The results show that TensorMask is close to Mask R-CNN both quantitatively and qualitatively.

Chinese teenager ×2

Chinese researchers are indispensable behind this new research.

The first author of the paper is a Chinese teenager named Chen Xinlei, who currently works at Facebook and has a brilliant research experience along the way.

When Chen Xinlei studied computer science at Zhejiang University, he was taught by Professor Cai Deng. After graduating from undergraduate school, he went to CMU to pursue a doctorate degree, under the tutelage of Professor Abhinav Gupta, mainly studying computer vision.

Before graduating with a Ph.D., Chen Xinlei interned in the Google Cloud AI department, where he was in the project team of two big names, Fei-Fei Li and Jia Li.

On Chen Xinlei’s personal homepage, we can see that many of the papers he participated in have been accepted by top conferences.

Everyone should be familiar with the third author He Kaiming. As the main proposer of Mask R-CNN, He Kaiming won the best paper award at top conferences three times.

He Kaiming was the top scorer in the Guangdong College Entrance Examination and was recommended to Tsinghua University. After graduating from undergraduate studies, He Kaiming entered the Chinese University of Hong Kong for postgraduate studies. During this period, he continued to participate in research at Microsoft Research Asia and currently works at Facebook.

The second author of the paper, Ross Girshick, and the fourth author, Piotr Dollar, are colleagues of Chen Xinlei and He Kaiming at Facebook. Top conference papers such as Mask R-CNN and Focal Loss for Dense Object Detection are all research conducted by the three of them in collaboration.

It is very good to have friends like this.

Portal

Paper TensorMask: A Foundation for Dense Object Segmentation:
https://arxiv.org/abs/1903.12174

The paper states that the research results will be open source soon~

Worth looking forward to.