• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer

Learn OpenCV

OpenCV, PyTorch, Keras, Tensorflow examples and tutorials

  • Home
  • Getting Started
    • Installation
    • PyTorch
    • Keras & Tensorflow
    • Resource Guide
  • Courses
    • Opencv Courses
    • CV4Faces (Old)
  • Resources
  • AI Consulting
  • About

Gaze Tracking

Satya Mallick
November 19, 2019 Leave a Comment
Eyes Tracking

November 19, 2019 By Leave a Comment

In today’s post, we are covering the topic of Gaze Estimation and Tracking.

I was invited to give a talk on the subject at a workshop on Eye Tracking for AR and VR organized by Facebook Research at the International Conference on Computer Vision (2019).

The slides are from that talk.

This post is for absolute beginners. We explain the problem of gaze estimation, current methods for collecting ground truth data, public datasets, and current methods for solving the gaze estimation/tracking problem. All the information is contained in the video. The text below simply links to some of the material presented above.

1. What is Gaze Estimation?

Simply put, in gaze estimation / tracking, we want to find where a person is looking.

There are two setups

1.1 Screen Based (or Remote ) Gaze Tracking

Screen based gaze tracking

In this setup, the person is looking at a screen (monitor) and we are interested in finding out the x and y coordinates of location on the screen where they are looking.

1.2 Gaze Tracking in Wearable Devices

In this setup, the person whose gaze we are interested in tracking wears a device (glasses). This device contains a camera, called the scene camera that records what the person is looking at.

The goal is to find the x and y coordinates of the person’s gaze in the scene camera.


Want to learn Deep Learning and Computer Vision in depth? OpenCV (in collaboration with LearnOpenCV) is offering 3 Computer Vision courses.

I want to know more

2. Gaze Tracking Resources

In this section, we will link to resources useful for gaze tracking research and development.

2.1 Research Papers and Algorithms

The literature on gaze tracking is vast. If you are interested in getting started, we refer you the recent survey paper that does a very good job of reviewing existing techniques.

A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms

The above survey paper was published in 2017. Since, then there was been additional papers of which the following are noteworthy

Deep Multitask Gaze Estimation with a Constrained Landmark-Gaze Mode

Recurrent CNN for 3D Gaze Estimation using Appearance and Shape Cues

Quite interestingly, there was a yet another workshop on gaze tracking at ICCV 2019 organized by Microsoft. The papers in the workshop are worth reviewing.

2.2 Gaze Tracking Products

Tobii sells many commercial wearable and screen trackers. You can find the complete product listing here.

https://www.tobiipro.com/product-listing/

We also have an open source option sold by Pupil Labs.

https://pupil-labs.com/

2.3 Datasets

The biggest challenge in eye gaze estimation research is the availability of good public datasets.

Fortunately, in recent years real as well as synthetic datasets have been released that are crucial to push research forward.

2.3.1 Synthetic Datasets for Gaze Tracking

SYNTHESEYES (2015) by University of Cambridge

UNITYEYES (2016) by University of Cambridge

NVGaze (2019) by NVIDIA

2.3.2 Real Datasets for Gaze Tracking

UTMULTIVIEW (2013) by The University of Tokyo

EYEDIAP (2014) by Idiap Research Institute

MPIIGAZE (2015) by MPI Informatik.

GAZECAPTURE (2016) by University of Georgia, Massachusetts Institute of Technology, and MPI Informatik

RT-GENE (2018) by Imperial College London.

OpenEDS : Open Eye Dataset by Facebook

Subscribe

If you liked this article, please subscribe to our newsletter. You will also receive a free Computer Vision Resource guide. In our newsletter, we share Computer Vision, Machine Learning and AI tutorials written in Python and C++ using OpenCV, Dlib, Keras, Tensorflow, CoreML, and Caffe.

Subscribe Now


Tags: eye tracking gaze estimation gaze tracking

Filed Under: Eyes, Tracking

About

I am an entrepreneur with a love for Computer Vision and Machine Learning with a dozen years of experience (and a Ph.D.) in the field.

In 2007, right after finishing my Ph.D., I co-founded TAAZ Inc. with my advisor Dr. David Kriegman and Kevin Barnes. The scalability, and robustness of our computer vision and machine learning algorithms have been put to rigorous test by more than 100M users who have tried our products. Read More…

Getting Started

  • Installation
  • PyTorch
  • Keras & Tensorflow
  • Resource Guide

Resources

Download Code (C++ / Python)

ENROLL IN OFFICIAL OPENCV COURSES

I've partnered with OpenCV.org to bring you official courses in Computer Vision, Machine Learning, and AI.
Learn More

Recent Posts

  • Making A Low-Cost Stereo Camera Using OpenCV
  • Optical Flow in OpenCV (C++/Python)
  • Introduction to Epipolar Geometry and Stereo Vision
  • Depth Estimation using Stereo matching
  • Classification with Localization: Convert any Keras Classifier to a Detector

Disclaimer

All views expressed on this site are my own and do not represent the opinions of OpenCV.org or any entity whatsoever with which I have been, am now, or will be affiliated.

GETTING STARTED

  • Installation
  • PyTorch
  • Keras & Tensorflow
  • Resource Guide

COURSES

  • Opencv Courses
  • CV4Faces (Old)

COPYRIGHT © 2020 - BIG VISION LLC

Privacy Policy | Terms & Conditions

We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.AcceptPrivacy policy