Videos



BetterFlow: Event-based Moving Object Detection and Tracking

Our paper presents a new, efficient approach to object tracking with asynchronous cameras, such as DVS or DAVIS. We present a novel event stream representation which enables us to utilize information about the dynamic (temporal) component of the event stream, and not only the spatial component, at every moment of time. The algorithm is capable of producing the motion-compensated event stream (effectively approximating egomotion) without using any form of external sensors in extremely low-light and noisy conditions, and without any form of feature tracking or explicit optical flow computation. We demonstrate our framework on the task of independent motion detection and tracking, where we use the temporal model inconsistencies to locate differently moving objects in challenging situations of very fast motion.

GapFlyt: Active Vision Based Minimalist Structure-less Gap Detection For Quadrotor Flight

In this paper, we propose this framework of bio-inspired perceptual design for quadrotors. We use this philosophy to design a minimalist sensori-motor framework for a quadrotor to fly though unknown gaps without a 3D reconstruction of the scene using only a monocular camera and onboard sensing. We successfully evaluate and demonstrate the proposed approach in many real-world experiments with different settings and window shapes, achieving a success rate of 85% at 2.5m/s even with a minimum tolerance of just 5cm.

SalientDSO: Bringing Attention to Direct Sparse Odometry

We introduce the philosophy of attention and fixation to visual odometry. Based on this philosophy, we develop Salient Direct Sparse Odometry, which brings the concept of attention and fixation based on visual saliency into Visual Odometry to achieve robust feature selection. We provide thorough quantitative and qualitative evaluations on ICL-NUIM and TUM monoVO dataset to demonstrate that using salient features improves the robustness and accuracy. We also collect and publicly release a new CVL dataset with cluttered scenes for mapping. We show the robustness of our features by very low drift visual odometry with as low as 40 features per frame.

Robot learns to cook

The video shows how our robot learns cooking by watching YOUTUBE videos.

The Robot Visual Learner

The video shows a demo,where the robot learns from a human how to mix a specific drink.

Action Grammar

This video explains the Computer Vision processes that are going on under the hood in order to interpret manipulation actions from a given set of frames.

Humanoid operates a refrigerator

This video shows how the robot can fetch milk and orange juice from the refrigerator.

A Robot for Cleaning

This video shows the robot to clean the table after being instructed by the user via gestures.

Humanoid Robot operating a Microwave

This video shows how the robot can use the microwave.

Pointing an object for Human Robot Interaction

The video illustrates human interaction with a robot. The person points at an object, and the robot has the 3D vision process to understand where the person is pointing to! The robot has on its left arm a camera, which it moves to see (it tracks) where the person is pointing to. In the upper left window, you can see the real-time detection of the object (the robot is pointing to) fitted by a cylinder. On the bottom right, you see what is recorded on the camera of the robot’s left arm.