Fast Image Downloader for Open Images V4

Last year, Google released a publicly available dataset called Open Images V4 which contains 15.4M annotated bounding boxes for over 600 object categories. It has 1.9M images and is largest among all existing datasets with object location annotations. The classes include a variety of objects in various categories. It covers classes varying from different kinds of musical instruments(e.g. organ, cello, piano etc.) to different kinds of aquatic animals(e.g. goldfish, crab, seahorse, oyster etc.) to various kinds of kitchenware(e.g. spoon, kitchen knife, frying pan, dishwasher) and so on. Below we show some classes of outdoor items.

All existing classes in Open Images can be seen as a dendrogram here.

The dataset can speed up many computer vision tasks by days or even months. For example, if we want to make an object detector for a single or multiple objects, we could download the images of those classes only along with their annotations and start our training process. In this post, we provide you a script that helps you download the images using multithreading, which can make the download process more than 10 times faster. It also generates the annotation files with the bounding box information for the objects instances in the images.

We present a table below which lists the number of train, validation and test instances for each class. You can also search for any objects of your interest in the table to see if there are enough number of instances to start your training.

Download data

Lets go through a step by step process of how we can download data for a selected set of classes.

Click here to download the source code to this post

Step 1:

First we will need to install awscli

sudo pip3 install awscli

Step 2:

Then we need to get the following OpenImages files

class-descriptions-boxable.csv – It contains a mapping of the class names used internally in the dataset to human interpretable names, e.g. /m/011k07 is Tortoise, /m/011q46kg is Container, /m/012074 is Magpie etc.

train-annotations-bbox.csv – It contains annotations of the object instances in the training images.

validation-annotations-bbox.csv – It contains annotations of the object instances in the validation images.

test-annotations-bbox.csv – It contains annotations of the object instances in the test images.

You could get the above images using the links above or using wget as shown below from the command line.

wget https://storage.googleapis.com/openimages/2018_04/class-descriptions-boxable.csv

wget https://storage.googleapis.com/openimages/2018_04/train/train-annotations-bbox.csv

wget https://storage.googleapis.com/openimages/2018_04/validation/validation-annotations-bbox.csv

wget https://storage.googleapis.com/openimages/2018_04/test/test-annotations-bbox.csv

Step 3:

Next, move the above .csv files to the same folder as the downloaded code and then use the following script to download the data. The example below downloads all the training instances for the three classes – Cheese, Ice cream and Cookie.

python3 downloadOI.py --classes 'Cheese,Ice_cream,Cookie' --mode train

Similarly, to get the validation or test images, we can set the mode argument to validation or test respectively.

When you do it for your own set of classes, make sure that if there is a space in the class name, it is replaced by an underscore(‘_’). In the above example, we replaced the ‘Ice cream’ class name with ‘Ice_cream’.

You can also add optional parameters to exclude certain kind of images by setting them explicitly to 0 in the command line.

occluded=0 to exclude occluded instances.

truncated=0 to exclude instances that are truncated at the boundary.

groupOf=0 to exclude instances that represent a group of objects together. These instances usually include a group of 5 or more objects of the same class physically touching or occluding each other, e.g. a bag of apples.

depiction=0 to exclude instances that are sketches or cartoons instead of picture of a real physical object.

inside=0 to exclude instances where the picture is taken from inside the object, e.g inside a car.

By default, all the above optional parameters are set to 1. So if you do not want to include the groupOf and inside instances, but want all other instances in your training, then the download command would be:

python3 downloadOI.py --classes 'Cheese,Ice_cream,Cookie' --mode train --groupOf=0 --inside=0

The directory structure of the download would be as follows:

– class-descriptions-boxable.csv
– train-annotations-bbox.csv
– validation-annotations-bbox.csv
– downloadOI.py
        – train
                – ClassA
                        – .jpg files
                        – .txt files
                – ClassB
                        – .jpg files
                        – .txt files
        – validation
                – ClassA
                        – .jpg files
                        – .txt files
                – ClassB
                        – .jpg files
                        – .txt files

Label Files

Open Images provides the top left corner and the bottom right corner of the bounding box of each object instance. The X and Y coordinates of these points are represented as proportions to the image width and height respectively.

The script writes the labels of all instances of the same class in the same image into a single text file. This label file is placed in the same directory as the image file, as shown in the directory structure above.

References

1. OpenImages: A public dataset for large-scale multi-label and multi-class image classification, 2017  

   Krasin I., Duerig T., Alldrin N., Ferrari V., Abu-El-Haija S., Kuznetsova A., Rom H., Uijlings J., Popov S., Kamali S., Malloci M., Pont-Tuset J., Veit A., Belongie S., Gomes V., Gupta A., Sun C., Chechik G., Cai D., Feng Z., Narayanan D., Murphy K.

2. The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale.  

   A. Kuznetsova, H. Rom, N. Alldrin, J. Uijlings, I. Krasin, J. Pont-Tuset, S. Kamali, S. Popov, M. Malloci, T. Duerig, and V. Ferrari. arXiv:1811.00982, 2018.