Devil is the Edges
Learning Semantic Boundaries from Noisy Annotations

David Acuna 1,2,3
Amlan Kar 2,3
Sanja Fidler1,2,3

2University of Toronto
3Vector Institute
CVPR, 2019 (Oral)

We tackle the problem of semantic boundary prediction, which aims to identify pixels that belong to object(class) boundaries. We notice that relevant datasets consist of a significant level of label noise, reflecting the fact that precise annotations are laborious to get and thus annotators trade-off quality with efficiency. We aim to learn sharp and precise semantic boundaries by explicitly reasoning about annotation noise during training. We propose a simple new layer and loss that can be used with existing learning-based boundary detectors. Our layer/loss enforces the detector to predict a maximum response along the normal direction at an edge, while also regularizing its direction. We further reason about true object boundaries during training using a level set formulation, which allows the network to learn from misaligned labels in an end-to-end fashion. Experiments show that we improve over the CASENet backbone network by more than 4% in terms of MF(ODS) and 18.61% in terms of AP, outperforming all current state-of-the-art methods including those that deal with alignment. Furthermore, we show that our learned network can be used to significantly improve coarse segmentation labels, lending itself as an efficient way to label new data.



David Acuna , Amlan Kar , Sanja Fidler

Devil is in the Edges: Learning Semantic Boundaries
from Noisy Annotations

CVPR, 2019. (to appear)


STEAL in a nutshell


Qualitative Results on the SBD Dataset

Qualitative Results on the Cityscapes Dataset

Coarse-to-fine Refinement

Make Better Segmentation Datasets with STEAL

Qualitative Results. Coarse-to-Fine on the coarsely annotated Cityscapes train extra set.