EKLT: Asynchronous, Photometric Feature Tracking using Events and Frames (IJCV'19)

This work presents EKLT, a feature tracking method that leverages the complementarity of event cameras and standard cameras to track visual features with low latency. Event cameras are novel sensors that output pixel-level brightness changes, called "events". They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the same scene pattern can produce different events depending on the motion direction, establishing event correspondences across time is challenging. By contrast, standard cameras provide intensity measurements (frames) that do not depend on motion direction. Our method extracts features on frames and subsequently tracks them asynchronously using events, thereby exploiting the best of both types of data: the frames provide a photometric representation that does not depend on motion direction and the events provide low-latency updates. In contrast to previous works, which are based on heuristics, this is the first principled method that uses raw intensity measurements directly, based on a generative event model within a maximum-likelihood framework. As a result, our method produces feature tracks that are more accurate than the state of the art, across a wide variety of scenes.
Reference:
Daniel Gehrig, Henri Rebecq, Guillermo Gallego, Davide Scaramuzza.
EKLT: Asynchronous, Photometric Feature Tracking using Events and Frames.
International Journal of Computer Vision (IJCV), Aug. 2019.
PDF: rpg.ifi.uzh.ch/...
Source code of the EKLT tracker: github.com/uzh...
Evaluation code: github.com/uzh...
Our research page on event based vision: rpg.ifi.uzh.ch/...
For event-camera datasets and event camera simulator, see here:
rpg.ifi.uzh.ch/...
Other resources on event cameras (publications, software, drivers, where to buy, etc.):
github.com/uzh...
Affiliations: D. Gehrig, H. Rebecq, G. Gallego and D. Scaramuzza are with the Robotics and Perception Group, Dept. of Informatics, University of Zurich, and Dept. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland rpg.ifi.uzh.ch/

[@roohollakhorrambakht8104]

It would be a dream come true joining you guys.