Robust and efficient semantic sensor registration for mobile robotics in unorganized, natural, scenes

Summary

Abstract

Advances in sensing and computing hardware have led to renewed interest in registration algorithms. In particular, the proliferation of 3D light detection and ranging (LIDAR) sensors and RGBD cameras and their use in robotic systems require efficient, robust, and accurate estimation algorithms for use in mapping, localization, and tracking tasks. Most modern approaches to autonomous driving require localizing and calibrating multiple LIDAR sensors, both of which are registration tasks. Meanwhile, tasks in the domain of indoor robotics require both localizing the robot and localizing objects of interest in the environment. The registration problem is that of trying to find the rigid body transformation between two measurements. This can include consecutive measurements (producing an odometry estimate), measurements from disparate points in time (such as for localization and mapping), and between different sensors (such as for calibrating multiple sensors on a platform).

Semantic detection and segmentation have similarly significantly progressed. Semantic inference on images and point clouds has shown increasing value in vision-based applications. The application of Convolutional Neural Networks (CNNs) has improved the computational efficiency of semantic segmentation techniques with superior performance in both indoor and outdoor benchmarks. Together with pose estimation techniques, multiple scenes can be segmented and combined to perform semantic mapping or object tracking; nevertheless, most semantic mapping and object tracking research has focused on performing pose estimation, and then semantic inference. So far, most research has not focused on joint semantic and metric estimation.

This thesis focuses on leveraging semantic inference to enable efficient and robust sensor registration. In robotics, semantic inference is increasingly used for downstream reasoning tasks. This thesis explores how that inference can be used in upstream task such as egomotion estimation, object pose estimation, and multisensor calibration. This work is based on improving the Iterative Closest Point (ICP) algorithm.

Our first contribution in this thesis explores how probabilistic semantic labels can be used in sensor registration. We present an approach that uses the Expectation Maximization (EM) technique to improve associations in the ICP framework. We also use an M-Estimator and optimize directly on the SE(3) manifold to improve the robustness. Our results on publicly available indoor and outdoor data sets show that semantics can help improve registration accuracy. For the second contribution, we add informative channels to the ICP framework to aid in object-level registration. This includes work on using sparse kernels to represent intensity and color channels for regularizing the registration problem, and work on curvature based alignment to improve object pose estimation. This technique extends registration algorithms beyond their purely geometric base. Our third contribution is a reformulation the registration problem as a mixed integer program (MIP). Most previous approaches to sensor registration use gradient-based optimization techniques. If the cost function used is nonconvex, they are prone to getting caught in local minima. The problem is reformulated as a MIP by linearizing the cost function and representing the data association as an integer valued variable.

This thesis focuses on developing robust and accurate registration techniques for mobile robotic applications. It presents results and proposed evaluation in the areas of indoor home robotics and autonomous driving, many of which are publicly available benchmark data sets. Sensor registration is a fundamental component of many robotic systems, and the advances proposed in this thesis have the potential to benefit many more aspects of perceptual systems.

Bibtex entry