Human-Machine collaboration for Large Scale Bounding Box Annotation

April 19, 2018
Versione stampabile

Time: h.15.30 am
Location: Room Levico, Via Sommarive 9 - Polo Ferrari 2


  • Jasper Uijlings, Google


Modern vision systems requires lots of manually annotated data for training, which is expensive to obtain. In this talk I will present our efforts at Google for efficiently annotating bounding boxes in images at scale using Human-Machine collaboration.

First I will talk about our work on Knowledge Transfer (CVPR 18), where we train object detectors on target classes from weakly supervised training images, helped by a set of source classes with bounding-box annotations. We use this and several other techniques to improve upon our previous work (CVPR 16) in which humans are asked to verify an automatically generated bounding box, circumventing the need for manual drawing. Then I will describe an improved method for manually drawing boxes, called Extreme Clicking (ICCV 17). Finally, I describe how we train an agent to automatically select between box verification questions and manual drawing, resulting in Intelligent Annotation Dialogs (CVPR 18).

About the Speaker

Jasper Uijlings is a research scientist at Google working with Vittorio Ferrari on human-machine collaboration for large scale image annotation for both object detection and image segmentation.

He received his PhD in 2011 at the University of Amsterdam (UvA) under supervision of Prof. Dr.Ir. A. Smeulders and Prof. Dr. Ir. R. Scha. During this period, he was part of the UvA team which successfully participated in the PASCAL VOC Challenges, winning the classification challenge in 2008, receiving honorable mentions from 2009-2011, and winning object detection in 2012, the final year of the competition.

In 2011 we won the ILSVRC object detection challenge. From 2011-2013 he was a researcher at the University of Trento, Italy, where he worked on real-time video classification and on combining vision and language. From 2014-2015 he worked at the University of Edinburgh with Prof.Dr. Vittorio Ferrari on object boundary detection, weakly supervised object localisation, and efficient annotation for object detection.

Contact Person for this Seminar: raffaella.bernardi [at] (Raffaella Bernardi)