Fast Image Downloader for Open Images V4

Last year, Google released a publicly available dataset called Open Images V4 which contains 15.4M annotated bounding boxes for over 600 object categories. It has 1.9M images and is largest among all existing datasets with object location annotations. The classes include a variety of objects in various categories. It covers classes varying from different kinds of musical instruments(e.g. organ, cello, piano etc.) to different kinds of aquatic animals(e.g. goldfish, crab, seahorse, oyster etc.) to various kinds of kitchenware(e.g. spoon, kitchen knife, frying pan, dishwasher) and so on. Below we show some classes of outdoor items.

All existing classes in Open Images can be seen as a dendrogram here.

The dataset can speed up many computer vision tasks by days or even months. For example, if we want to make an object detector for a single or multiple objects, we could download the images of those classes only along with their annotations and start our training process. In this post, we provide you a script that helps you download the images using multithreading, which can make the download process more than 10 times faster. It also generates the annotation files with the bounding box information for the objects instances in the images.

We present a table below which lists the number of train, validation and test instances for each class. You can also search for any objects of your interest in the table to see if there are enough number of instances to start your training.

Class Name	Train	Validation	Test
Clothing	1438128	8527	26531
Man	1418594	5654	17514
Tree	1051344	3209	10148
Human face	1037710	5170	15536
Person	1034721	13274	40861
Woman	767337	2865	9047
Footwear	744474	7189	21205
Window	503467	1650	5091
Flower	345296	5089	15040
Wheel	340639	11394	34604
Plant	267913	3808	11579
Car	248075	9381	28737
Human hair	234057	8594	26301
Human arm	208982	8341	25162
Human head	201633	7865	25080
Girl	197155	2420	7479
Building	178634	984	2915
Human body	175244	6769	20246
Mammal	156154	4349	13479
House	136152	246	822
Chair	132483	511	1535
Tire	122615	4181	13177
Suit	110848	321	857
Fashion accessory	91024	1026	3164
Food	88422	2736	8331
Boy	87555	600	2031
Table	85691	714	2279
Skyscraper	81261	57	232
Land vehicle	81108	2689	8480
Boat	79113	903	2672
Jeans	78473	396	1433
Human eye	77233	4304	13034
Human hand	75307	4123	12505
Human leg	71479	4093	13334
Toy	70963	437	1205
Tower	67945	82	264
Human nose	60142	4341	12718
Bicycle wheel	59521	733	2018
Glasses	57946	262	890
Dress	52999	567	1581
Vehicle	50959	2105	7064
Bird	47921	943	2751
Sports equipment	44900	3951	11992
Street light	44697	22	242
Human mouth	44197	2505	7424
Palm tree	42026	253	620
Book	41280	698	2147
Tableware	41086	285	936
Drink	40323	482	1427
Bottle	40188	340	979
Bicycle	40161	403	1158
Furniture	38527	646	1893
Snack	37374	708	2173
Sculpture	34533	221	653
Flag	29246	87	252
Dog	28675	1930	5818
Dessert	27407	645	2092
Microphone	27272	110	272
Fruit	26236	1905	6215
Jacket	25957	136	461
Guitar	25896	114	357
Fast food	24991	492	1599
Drum	24818	109	352
Sunglasses	23996	114	390
Poster	23566	107	346
Fish	23195	564	1422
Baked goods	23010	1020	2907
Shelf	22899	191	563
Houseplant	22834	319	825
Flowerpot	22760	302	659
Airplane	21285	1027	3272
Sports uniform	19396	315	1135
Door	19256	165	524
Vegetable	18621	496	1679
Human ear	17774	870	2611
Animal	17442	290	882
Shorts	16981	134	443
Musical instrument	16503	178	525
Helmet	16502	440	1275
Bicycle helmet	15952	147	390
Duck	15451	119	294
Wine	15400	193	388
Cat	15183	381	1095
Auto part	13586	2898	8845
Balloon	13505	117	455
Motorcycle	13382	173	530
Horse	13368	392	1144
Hat	13245	201	557
Train	13050	91	315
Wine glass	12934	119	356
Truck	12135	311	969
Rose	12053	309	899
Picture frame	11957	57	158
Bus	11927	92	353
Football helmet	11705	130	375
Desk	11693	48	136
Cattle	11603	170	450
Bee	11401	84	279
Tie	10545	84	271
Hiking equipment	10505	93	458
Butterfly	10127	74	265
Swimwear	10079	149	574
Billboard	9823	29	50
Goggles	9636	101	218
Beer	9565	118	332
Laptop	9327	61	199
Cabinetry	9191	188	451
Marine invertebrates	9112	68	217
Insect	8981	210	717
Trousers	8481	143	495
Goose	8436	55	116
Dairy	8146	308	970
Strawberry	7944	326	774
Vehicle registration plate	7852	512	1570
Van	7720	151	500
Shirt	7465	57	185
Traffic light	7426	49	131
Bench	7229	0	0
Umbrella	7204	30	97
Sun hat	6979	52	205
Paddle	6951	253	699
Tent	6907	30	117
Ball	6845	266	902
Sunflower	6815	55	124
Coat	6523	55	240
Lavender	6472	87	326
Doll	6442	77	188
Camera	6404	132	338
Mobile phone	6365	137	454
Tomato	6254	216	722
Office supplies	6198	76	233
Orange	6195	175	839
Pumpkin	6150	49	157
Traffic sign	6112	28	72
Computer monitor	6112	57	215
Stairs	5981	37	122
Candle	5886	75	134
Pastry	5852	48	170
Cake	5784	326	878
Roller skates	5476	295	723
Lantern	5429	27	60
Plate	5416	67	214
Box	5364	94	256
Coffee cup	5327	160	538
Coffee table	5314	41	104
Bookcase	5307	100	301
Watercraft	5202	155	510
Football	5097	64	220
Office building	4986	45	103
Maple	4923	13	54
Curtain	4872	48	170
Kitchen appliance	4662	76	234
Muffin	4608	82	310
Canoe	4543	82	292
Computer keyboard	4542	42	173
Swan	4523	49	136
Bowl	4507	51	153
Mushroom	4497	126	465
Cocktail	4458	148	406
Drawer	4414	184	448
Castle	4310	12	71
Couch	4259	44	104
Christmas tree	4243	53	83
Taxi	4199	91	104
Penguin	4197	85	161
Cookie	4158	184	636
Apple	3898	102	334
Swimming pool	3881	142	354
Deer	3867	132	449
Porch	3854	33	100
Bread	3846	277	911
Bowling equipment	3846	98	339
Television	3789	39	124
Fountain	3691	19	63
Lifejacket	3678	103	273
Lamp	3663	70	116
Fedora	3660	51	175
Bed	3563	114	328
Beetle	3523	47	163
Pillow	3508	61	264
Ski	3505	78	431
Carnivore	3501	200	625
Platter	3462	83	198
Sheep	3438	89	273
Elephant	3272	56	232
Human beard	3157	153	481
Boot	3132	163	472
High heels	3124	119	277
Countertop	3113	61	143
Salad	3088	213	605
Cowboy hat	3068	24	142
Seafood	3063	226	689
Chicken	3059	65	227
Coin	3042	93	245
Monkey	3026	195	543
Helicopter	3023	134	424
Tin can	2988	42	188
Weapon	2960	155	388
Sandal	2938	181	393
Juice	2838	174	512
Ice cream	2834	118	395
Violin	2828	35	98
Saucer	2819	73	248
Grape	2787	153	447
Cart	2755	75	200
Bronze sculpture	2748	51	101
Necklace	2735	60	162
Bathroom accessory	2678	53	132
Parachute	2672	121	286
Skull	2661	24	91
Surfboard	2594	65	189
Rifle	2540	110	315
Baseball glove	2529	31	89
Handbag	2495	60	188
Vase	2468	56	171
Cosmetics	2394	50	151
Parrot	2388	82	283
Coffee	2384	124	353
Scarf	2303	33	68
Mug	2272	24	107
Candy	2261	116	337
Lily	2252	124	496
Human foot	2237	199	467
Luggage and bags	2220	35	99
Goldfish	2204	38	129
Kitchen & dining room table	2127	44	157
Lizard	2120	95	301
French fries	2114	75	215
Sushi	2088	74	186
Home appliance	2086	99	275
Barrel	2086	73	165
Harbor seal	2084	50	157
Tool	2075	41	137
Goat	2075	76	245
Jellyfish	2064	91	133
Medical equipment	2060	84	207
Spider	2033	63	224
Pizza	2008	95	217
Cello	2004	38	85
Tortoise	1998	53	178
Squirrel	1940	40	94
Watch	1903	26	77
Aircraft	1898	186	556
Studio couch	1889	51	141
Gondola	1868	25	74
Egg	1865	144	576
Moths and butterflies	1857	16	41
Shrimp	1856	116	266
Sea lion	1823	54	170
Convenience store	1817	65	171
Light bulb	1816	37	47
Skateboard	1810	72	203
Waste container	1807	19	103
Musical keyboard	1771	46	151
Kite	1766	30	75
Lemon	1756	171	425
Marine mammal	1746	47	157
Bull	1736	107	289
Dinosaur	1721	28	57
Falcon	1717	37	156
Tank	1716	58	161
Spoon	1709	50	162
Pen	1705	49	149
Eagle	1704	50	201
Tap	1695	46	108
Brassiere	1694	83	246
Fork	1687	45	143
Owl	1663	30	91
Lion	1653	62	142
Sparrow	1651	63	163
Sink	1648	54	119
Rabbit	1641	62	220
Pig	1613	69	201
Banana	1612	22	133
Frog	1608	41	122
Teddy bear	1587	30	125
Mirror	1572	21	84
Invertebrate	1568	124	376
Antelope	1568	85	385
Cheese	1560	127	323
Trumpet	1546	46	123
Dolphin	1532	87	237
Chest of drawers	1526	87	196
Lighthouse	1518	8	34
Sofa bed	1501	55	150
Dragonfly	1490	16	65
Hamburger	1486	80	206
Wheelchair	1464	107	288
Carrot	1456	50	155
Tripod	1446	26	103
Earrings	1446	25	76
Giraffe	1431	25	73
Sock	1425	15	136
Snake	1378	64	241
Piano	1374	82	215
Cupboard	1353	85	218
Lipstick	1343	37	99
Tea	1342	47	136
Camel	1340	30	87
Shellfish	1287	50	128
Grapefruit	1283	62	344
Tiger	1260	21	57
Skirt	1259	15	61
Headphones	1255	35	97
Stool	1254	30	85
Horn	1239	36	110
Baseball bat	1228	19	48
Clock	1222	23	60
Backpack	1216	28	70
Saxophone	1208	37	97
Glove	1198	54	158
Cucumber	1194	100	259
Sandwich	1157	90	240
Bear	1137	50	163
Sea turtle	1132	36	155
Broccoli	1128	32	182
Nightstand	1125	24	75
Zebra	1120	44	123
Mule	1117	41	97
Toilet	1099	28	61
Zucchini	1098	50	164
Cannon	1087	20	66
Crocodile	1069	52	108
Wall clock	1067	18	42
Bust	1060	18	29
Crab	1041	66	162
Oyster	1038	106	159
Whale	1014	42	124
Mixing bowl	1005	25	65
Whiteboard	1003	16	37
Ladder	994	20	65
Plastic bag	986	9	19
Tennis racket	985	21	102
Barge	983	4	11
Tablet computer	975	9	37
Tart	973	26	151
Accordion	955	24	79
Miniskirt	954	28	89
Trombone	953	17	61
Snowboard	944	16	53
Snail	943	39	136
Doughnut	930	49	192
Ant	925	28	76
Pear	923	18	111
Rocket	918	13	88
Billiard table	912	28	85
Caterpillar	884	24	104
Panda	882	20	51
Coconut	874	21	91
Mouse	857	34	165
Knife	850	77	216
Table tennis racket	849	29	110
Watermelon	844	32	126
Alpaca	829	34	121
Leopard	811	40	125
Bell pepper	802	67	219
Kangaroo	778	32	82
Pancake	775	63	102
Snowman	770	4	30
Pasta	769	118	327
Peach	756	64	280
Otter	752	43	146
Door handle	751	19	79
Willow	735	2	5
Turkey	734	23	72
Ladybug	734	23	64
Computer mouse	733	24	84
Wok	730	24	36
Handgun	727	24	81
Rhinoceros	724	21	70
Cheetah	715	29	72
Dice	714	52	119
Fireplace	711	35	62
Waffle	710	25	90
Radish	688	21	67
Crown	687	19	50
Hippopotamus	685	13	54
Mechanical fan	681	20	65
Taco	677	49	194
Pomegranate	677	12	62
Polar bear	664	35	93
Volleyball	661	18	111
Closet	661	10	44
Pineapple	660	25	110
Kettle	657	26	73
Washing machine	655	39	121
Bat	655	6	28
Sombrero	651	7	20
Brown bear	647	13	77
Ostrich	640	34	137
Bagel	640	43	111
Starfish	639	21	81
Oven	637	21	53
Teapot	632	37	78
Loveseat	631	21	50
Suitcase	630	30	79
Shark	625	52	99
Chopsticks	617	14	36
Swim cap	615	37	140
Missile	603	22	75
Potato	599	56	128
Lobster	597	30	95
Ipod	595	29	61
Golf cart	595	8	32
Bow and arrow	594	21	45
Refrigerator	592	20	82
Jug	590	23	76
Jaguar	586	35	95
Shotgun	580	26	66
Reptile	578	34	93
Window blind	570	13	51
Sword	567	23	89
Raven	567	32	107
Segway	565	34	132
Fox	565	38	82
Kitchen utensil	549	57	142
Hamster	546	32	140
Bathtub	545	15	59
Jet ski	543	18	32
Gas stove	526	17	49
Scoreboard	517	9	16
Woodpecker	515	14	40
Beehive	511	27	57
Tennis ball	502	8	39
Nail	491	0	2
Rays and skates	485	26	83
Microwave oven	485	20	46
Hot dog	482	12	48
Plumbing fixture	481	51	125
Ceiling fan	478	11	40
Infant bed	462	18	62
Seat belt	461	3	8
Sewing machine	453	25	74
Croissant	447	40	78
Ambulance	447	13	67
Bidet	440	21	49
Cabbage	435	24	101
Golf ball	434	12	56
Corded phone	433	6	18
Fire hydrant	432	14	57
Mango	429	12	26
Picnic basket	425	12	26
Red panda	423	12	62
Belt	422	13	32
Koala	418	14	49
Dumbbell	413	13	32
Tiara	411	12	46
Personal care	409	11	18
Scissors	399	1	6
Organ	398	21	63
Stop sign	394	13	33
Canary	387	13	32
Asparagus	387	19	53
Honeycomb	383	11	44
Raccoon	381	23	78
Toilet paper	377	3	11
Frying pan	377	11	16
Artichoke	376	26	63
Squash	375	20	47
Filing cabinet	374	24	82
Dagger	370	34	101
Snowmobile	366	8	70
Limousine	366	31	69
Perfume	363	19	46
Popcorn	362	19	91
Flute	362	14	31
Bathroom cabinet	358	2	16
Kitchen knife	350	29	83
Pitcher	347	14	41
Towel	338	7	29
Stationary bicycle	338	4	37
Cake stand	337	3	7
Punching bag	336	0	10
Ratchet	327	0	0
Balance beam	326	9	48
Coffeemaker	323	18	50
Common fig	317	24	74
Seahorse	314	22	60
Wood-burning stove	300	19	34
Snowplow	300	19	50
Rugby ball	294	7	28
Pretzel	294	21	35
Drinking straw	292	4	17
Power plugs and sockets	290	20	44
Racket	281	34	99
Centipede	280	11	59
Telephone	274	16	26
Submarine sandwich	273	29	66
Worm	270	2	19
Dog bed	266	3	8
Banjo	264	8	13
Printer	263	19	48
Burrito	262	34	121
Hedgehog	261	22	83
Blue jay	259	21	60
Adhesive tape	255	1	1
Wine rack	254	7	18
Ruler	253	15	35
Flying disc	249	0	3
Treadmill	247	7	20
Wardrobe	238	8	42
Lynx	237	13	48
Remote control	236	7	26
Shower	235	9	32
Blender	235	10	30
Harp	231	18	50
Porcupine	229	17	87
Guacamole	224	21	65
Squid	221	8	23
Toothbrush	219	16	20
Mixer	216	9	40
Milk	214	14	39
Cutting board	213	2	13
Harpsichord	212	15	33
Paper towel	210	2	7
Calculator	210	11	51
Parking meter	209	2	3
Cat furniture	208	0	2
Turtle	205	8	20
Wrench	204	1	3
Scorpion	204	17	49
Drill	203	1	5
Digital clock	199	7	39
Unicycle	194	2	14
Training bench	194	2	10
Food processor	192	3	27
Whisk	180	0	1
Salt and pepper shakers	180	4	5
Envelope	177	4	12
Stretcher	174	6	13
Alarm clock	169	10	27
Beaker	168	0	6
Cantaloupe	166	16	29
Oboe	162	6	28
Briefcase	162	4	17
Isopod	154	8	7
Diaper	152	1	3
Crutch	150	0	2
Axe	148	0	2
Magpie	145	22	39
Tick	143	0	1
Scale	139	5	14
Hammer	139	1	0
Pencil case	132	1	5
Cricket ball	132	3	7
Spice rack	131	3	19
Chainsaw	130	2	7
Syringe	127	0	19
Slow cooker	125	1	5
Cream	123	16	69
Binoculars	123	6	16
Serving tray	118	1	8
Tree house	110	4	19
Jacuzzi	103	16	28
Light switch	97	10	7
Dishwasher	92	2	3
Flashlight	88	6	23
Screwdriver	85	1	0
Ring binder	84	0	0
Submarine	81	10	21
Face powder	80	3	17
Stethoscope	78	1	9
Soap dispenser	78	0	1
Horizontal bar	75	6	9
Measuring cup	74	0	1
Cassette deck	74	9	32
Toaster	73	1	4
Hand dryer	70	0	1
Spatula	64	4	2
Stapler	59	0	1
Skunk	56	3	23
Armadillo	56	0	0
Ladle	54	0	0
Eraser	53	0	1
Cooking spray	45	0	0
Winter melon	43	3	8
Chime	41	0	1
Harmonica	38	0	5
Band-aid	36	0	0
Indoor rower	35	0	6
Heater	35	0	0
Chisel	33	1	0
Waffle iron	31	0	0
Fax	28	4	3
Hair dryer	27	0	3
Cocktail shaker	27	0	0
Pencil sharpener	21	0	0
Bottle opener	21	0	1
Torch	20	2	14
Pizza cutter	20	0	0
Facial tissue holder	20	0	0
Pressure cooker	14	1	1
Grinder	14	0	0
Humidifier	11	0	0
Maracas	10	0	0
Hair spray	10	0	0
Bomb	8	0	0
Can opener	7	0	0
Paper cutter	4	0	0
Kitchenware	0	0	0
Container	0	0	0

Download data

Lets go through a step by step process of how we can download data for a selected set of classes.

Step 1:

First we will need to install awscli

sudo pip3 install awscli

Step 2:

Then we need to get the following OpenImages files

class-descriptions-boxable.csv – It contains a mapping of the class names used internally in the dataset to human interpretable names, e.g. /m/011k07 is Tortoise, /m/011q46kg is Container, /m/012074 is Magpie etc.

train-annotations-bbox.csv – It contains annotations of the object instances in the training images.

validation-annotations-bbox.csv – It contains annotations of the object instances in the validation images.

test-annotations-bbox.csv – It contains annotations of the object instances in the test images.

You could get the above images using the links above or using wget as shown below from the command line.

wget https://storage.googleapis.com/openimages/2018_04/class-descriptions-boxable.csv

wget https://storage.googleapis.com/openimages/2018_04/train/train-annotations-bbox.csv

wget https://storage.googleapis.com/openimages/2018_04/validation/validation-annotations-bbox.csv

wget https://storage.googleapis.com/openimages/2018_04/test/test-annotations-bbox.csv 
 

Step 3:

Next, move the above .csv files to the same folder as the downloaded code and then use the following script to download the data. The example below downloads all the training instances for the three classes – Cheese, Ice cream and Cookie.

python3 downloadOI.py --classes 'Cheese,Ice_cream,Cookie' --mode train
 

Similarly, to get the validation or test images, we can set the mode argument to validation or test respectively.

When you do it for your own set of classes, make sure that if there is a space in the class name, it is replaced by an underscore(‘_’). In the above example, we replaced the ‘Ice cream’ class name with ‘Ice_cream’.

You can also add optional parameters to exclude certain kind of images by setting them explicitly to 0 in the command line.

occluded=0 to exclude occluded instances.

truncated=0 to exclude instances that are truncated at the boundary.

groupOf=0 to exclude instances that represent a group of objects together. These instances usually include a group of 5 or more objects of the same class physically touching or occluding each other, e.g. a bag of apples.

depiction=0 to exclude instances that are sketches or cartoons instead of picture of a real physical object.

inside=0 to exclude instances where the picture is taken from inside the object, e.g inside a car.

By default, all the above optional parameters are set to 1. So if you do not want to include the groupOf and inside instances, but want all other instances in your training, then the download command would be:

python3 downloadOI.py --classes 'Cheese,Ice_cream,Cookie' --mode train --groupOf=0 --inside=0
 

The directory structure of the download would be as follows:

– class-descriptions-boxable.csv
– train-annotations-bbox.csv
– validation-annotations-bbox.csv
– downloadOI.py
        – train
                – ClassA
                        – .jpg files
                        – .txt files
                – ClassB
                        – .jpg files
                        – .txt files
        – validation
                – ClassA
                        – .jpg files
                        – .txt files
                – ClassB
                        – .jpg files
                        – .txt files

Label Files

Open Images provides the top left corner and the bottom right corner of the bounding box of each object instance. The X and Y coordinates of these points are represented as proportions to the image width and height respectively.

The script writes the labels of all instances of the same class in the same image into a single text file. This label file is placed in the same directory as the image file, as shown in the directory structure above.

Subscribe & Download Code

If you liked this article and would like to download code (C++ and Python) and example images used in this post, please subscribe to our newsletter. You will also receive a free Computer Vision Resource Guide. In our newsletter, we share OpenCV tutorials and examples written in C++/Python, and Computer Vision and Machine Learning algorithms and news.

References

1. OpenImages: A public dataset for large-scale multi-label and multi-class image classification, 2017

   Krasin I., Duerig T., Alldrin N., Ferrari V., Abu-El-Haija S., Kuznetsova A., Rom H., Uijlings J., Popov S., Kamali S., Malloci M., Pont-Tuset J., Veit A., Belongie S., Gomes V., Gupta A., Sun C., Chechik G., Cai D., Feng Z., Narayanan D., Murphy K.

2. The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale.

   A. Kuznetsova, H. Rom, N. Alldrin, J. Uijlings, I. Krasin, J. Pont-Tuset, S. Kamali, S. Popov, M. Malloci, T. Duerig, and V. Ferrari.
   arXiv:1811.00982, 2018.