Instance-aware Semantic Segmentation
Instance-aware Semantic Segmentation
Problem
The task requires both classification and segmentation of object instances.
Evaluation
Dataset
PASCAL VOC 2012 segmentation val subset
1449 images.
training data: 10582 training images and annotations are from the train subset and SBD.
PASCAL VOC 2012 SDS val subset
5732 images, the same 5623 training images in the train subset.
Microsoft COCO
80 object classes.
trainval subset: 80k + 40k images.
Result
PASCAL VOC 2012 segmentation val
| IoU threshold | 0.6 | 0.7 | 0.8 | 0.9 |
|---|---|---|---|---|
| PFN | 51.3 | 42.5 | 31.2 | 15.7 |
| MPA 1-scale | 54.6 | 45.9 | 34.3 | 17.3 |
| MPA 3-scale | 56.6 | 47.4 | 36.1 | 18.5 |
PASCAL VOC 2012 SBD val
| Method | \(mAP^r\)@0.5 (%) | \(mAP^r_{vol}\) |
|---|---|---|
| SDS | 49.7 | 41.4 |
| Hypercolumn | 56.5 | - |
| Hypercolumn-rescore | 60.0 | - |
| CFM | 60.7 | - |
| MPA 1-scale | 55.5 | 48.3 |
| MPA 3-scale-rescore | 61.8 | 52.0 |
| method | \(mAP^r\)@0.5 (%) | \(mAP^r\)@0.7 (%) |
|---|---|---|
| \(O^2P\) | 25.2 | - |
| SDS | 49.7 | 25.3 |
| naive MNC | 59.1 | 36.0 |
| Hypercolumn | 56.5 | 37.0 |
| Hypercolumn-rescore | 60.0 | 40.4 |
| CFM | 60.7 | 39.6 |
| MNC | 63.5 | 41.5 |
| InstFCN + MNC | 61.5 | 43.0 |
| InstFCN + R-FCN | 62.7 | 41.5 |
| FCIS (translation invariant) | 52.5 | 38.5 |
| FCIS (separate score maps) | 63.9 | 49.7 |
| FCIS | 65.7 | 52.1 |
Microsoft COCO
| Method | backbone | \(mAP^r\)@[0.5:0.95] (%) | \(mAP^r\)@0.5 (%) |
|---|---|---|---|
| MNC | VGG-16 | 19.5 | 39.7 |
| MNC | ResNet-101-C4 | 24.6 | 44.3 |
| FAIRCNN (2015) | 25.0 | 45.6 | |
| MNC+++ (2015) | 28.4 | 51.6 | |
| G-RMI (2016) | 33.8 | 56.9 | |
| FCIS baseline | ResNet-101-C5-dilated | 29.2 | 49.5 |
| +multi-scale testing | ResNet-101-C5-dilated | 32.0 | 51.9 |
| +horizontal flip | ResNet-101-C5-dilated | 32.7 | 52.7 |
| +multi-scale training | ResNet-101-C5-dilated | 33.6 | 54.5 |
| +ensemble | ResNet-101-C5-dilated | 37.6 | 59.9 |
| Mask R-CNN | ResNet-101-C4 | 33.1 | 54.9 |
| Mask R-CNN | ResNet-101-FPN | 35.7 | 58.0 |
| Mask R-CNN | ResNeXt-101-FPN | 37.1 | 60.0 |
Reference
| Method | Year | Conference | Reference Paper |
|---|---|---|---|
| SDS | 2014 | ECCV | Simultaneous Detection and Segmentation |
| Hypercolumn | 2015 | CVPR | Hypercolumns for Object Segmentation and Fine-grained Localization |
| CFM | 2015 | CVPR | Convolutional Feature Masking for Joint Object and Stuff Segmentation |
| MNC | 2016 | CVPR | Instance-aware Semantic Segmentation via Multi-task Network Cascades |
| MPA-SDS | 2016 | CVPR | Multi-scale Patch Aggregation (MPA) for Simultaneous Detection and Segmentation |
| InstFCN | 2016 | ECCV | Instance-sensitive Fully Convolutional Networks |
| FCIS | 2017 | CVPR | Fully Convolutional Instance-aware Semantic Segmentation |
| Mask R-CNN | 2017 | ICCV | Mask R-CNN |
