Visual Search and Navigation on Semantic SLAM
Published in Intelligent Robot Lab, National Taiwan University, 2019
Combined ORB SLAM and semantic segmentation on each frame into a real-time system that builds semantic point cloud and designed an effective algorithm to localize target objects in 3D maps.
Abstract
I integrated 2D grid mapping as well as navigation function on to my system done last semester, which is a real-time visual search system based on the octomap constructed by ORB slam2 and semantic segmentation. The whole system runs on Robot Operating System (ROS), using the Asus Xtion camera as the only sensor. Adapting the RGB-D version of ORB slam2, the system is able to estimate camera poses accurately. Using a PSPNet model first trained on ADE20K dataset and then fine-tuned on SUNRGBD dataset, a semantic 3D octomap was constructed. The octomap was projected into 2D grid map for navigation. The navigation function was appended to the system. The target coordinate was generated by my self-defined localization algorithm, and the path planner is conducted by move_base. Although the system requires improvement for accuracy, it is now a complete visual search engine with SLAM and navigation.
Report
I worked on this project for two semesters. As a result, there are two reports according to each semester.