Software & Datasets

GapFlyt

We propose a framework of bio-inspired perceptual design for quadrotors. We use this philosophy to design a minimalist sensori-motor framework for a quadrotor to fly though unknown gaps without a 3D reconstruction of the scene using only a monocular camera and onboard sensing. We successfully evaluate and demonstrate the proposed approach in many real-world experiments with different settings and window shapes, achieving a success rate of 85% at 2.5m/s even with a minimum tolerance of just 5cm. To our knowledge, this is the first paper which addresses the problem of gap detection of an unknown shape and location with a monocular camera and onboard sensing.

Authors: Nitin J. Sanket, Chahat Deep Singh and Kanishka Ganguly
[Code: GitHub]

SymSeg: Model-Free Object Segmentation in Point Clouds Using the Symmetry Constraint

SymSeg is a model-free method for segmenting objects and recovering their symmetries from multiple-view 3D pointclouds. The algorithm takes a pointcloud and an occupancy map of the scene as input and returns a set of object segmentation masks and corresponding symmetries.

Author: Aleksandrs Ecins
[Code: GitHub]

cilantro

cilantro is a lean and fast C++ library for working with point cloud data, with emphasis given to the 3D case. It includes efficient implementations for a variety of common operations, providing a clean API and attempting to minimize the amount of boilerplate code. The library is extensively templated, enabling operations on point data of arbitrary numerical type and dimensionality (where applicable) and featuring a modular/extensible design of the more complex procedures, while, at the same time, providing convenience aliases/wrappers for the most common cases.

Author: Konstantinos Zampogiannis
[Code: GitHub]

RGB-D affordance dataset

As robots begin to collaborate with humans in everyday workspaces, they will need to understand the functions of tools and their parts. To cut an apple or hammer a nail, robots need to not just know the tool's name, but they must localize the tool's parts and identify their functions. Intuitively, the geometry of a part is closely related to its possible functions, or its affordances. Therefore, we propose two approaches for learning affordances from local shape and geometry primitives: 1) superpixel based hierarchical matching pursuit (S-HMP); and 2) structured random forests (SRF). We validate the effectiveness of our approaches on the Cornell Grasping Dataset for detecting graspable regions, and achieve state-of-the-art performance.

Author: Austin Myers
[Code and Datset: Project Page]

LightNet: A Versatile, Standalone Matlab-based Environment for Deep Learning

LightNet is a lightweight, versatile and purely Matlab-based deep learning framework. The aim of the design is to provide an easy-to-understand, easy-to-use and efficient computational platform for deep learning research. The implemented framework supports major deep learning architectures such as the Multilayer Perceptron Networks (MLP), Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN). LightNet supports both CPU and GPU for computation and the switch between them is straightforward. Different applications in computer vision, natural language processing and robotics are demonstrated as experiments.

Author: Chengxi Ye
[Code: GitHub]

Event based Moving Object Dataset

A new publicly available event dataset (Extreme Event Dataset or EED) with multiple moving objects in challenging conditions. The scenes consist of multiple moving objects, low lighting conditions and extreme light variation including flashing strobe lights. A thorough quantitative and qualitative evaluation of the pipeline on the aforementioned dataset.

Authors: Anton Mitrokhin and Chethan M. Parameshwara
[ Dataset]