Learning intelligent dialogs for bounding box annotation

Uijlings J, Konyushkova K, Lampert C, Ferrari V. 2018. Learning intelligent dialogs for bounding box annotation. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVF: Conference on Computer Vision and Pattern Recognition, 9175–9184.


Conference Paper | Published | English

Scopus indexed
Author
Uijlings, Jasper; Konyushkova, Ksenia; Lampert , ChristophISTA ; Ferrari, Vittorio
Department
Abstract
We introduce Intelligent Annotation Dialogs for bounding box annotation. We train an agent to automatically choose a sequence of actions for a human annotator to produce a bounding box in a minimal amount of time. Specifically, we consider two actions: box verification [34], where the annotator verifies a box generated by an object detector, and manual box drawing. We explore two kinds of agents, one based on predicting the probability that a box will be positively verified, and the other based on reinforcement learning. We demonstrate that (1) our agents are able to learn efficient annotation strategies in several scenarios, automatically adapting to the image difficulty, the desired quality of the boxes, and the detector strength; (2) in all scenarios the resulting annotation dialogs speed up annotation compared to manual box drawing alone and box verification alone, while also outperforming any fixed combination of verification and drawing in most scenarios; (3) in a realistic scenario where the detector is iteratively re-trained, our agents evolve a series of strategies that reflect the shifting trade-off between verification and drawing as the detector grows stronger.
Publishing Year
Date Published
2018-12-17
Proceedings Title
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Page
9175-9184
Conference
CVF: Conference on Computer Vision and Pattern Recognition
Conference Location
Salt Lake City, UT, United States
Conference Date
2018-06-18 – 2018-06-23
eISSN
IST-REx-ID

Cite this

Uijlings J, Konyushkova K, Lampert C, Ferrari V. Learning intelligent dialogs for bounding box annotation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE; 2018:9175-9184. doi:10.1109/cvpr.2018.00956
Uijlings, J., Konyushkova, K., Lampert, C., & Ferrari, V. (2018). Learning intelligent dialogs for bounding box annotation. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9175–9184). Salt Lake City, UT, United States: IEEE. https://doi.org/10.1109/cvpr.2018.00956
Uijlings, Jasper, Ksenia Konyushkova, Christoph Lampert, and Vittorio Ferrari. “Learning Intelligent Dialogs for Bounding Box Annotation.” In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9175–84. IEEE, 2018. https://doi.org/10.1109/cvpr.2018.00956.
J. Uijlings, K. Konyushkova, C. Lampert, and V. Ferrari, “Learning intelligent dialogs for bounding box annotation,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, United States, 2018, pp. 9175–9184.
Uijlings J, Konyushkova K, Lampert C, Ferrari V. 2018. Learning intelligent dialogs for bounding box annotation. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVF: Conference on Computer Vision and Pattern Recognition, 9175–9184.
Uijlings, Jasper, et al. “Learning Intelligent Dialogs for Bounding Box Annotation.” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2018, pp. 9175–84, doi:10.1109/cvpr.2018.00956.
All files available under the following license(s):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]

Link(s) to Main File(s)
Access Level
OA Open Access

Export

Marked Publications

Open Data ISTA Research Explorer

Sources

arXiv 1712.08087

Search this title in

Google Scholar
ISBN Search