Both sides previous revision
Previous revision
Next revision
|
Previous revision
|
research:lsdslam [2015/03/12 08:43] engelj |
research:lsdslam [2015/07/27 22:44] (current) engelj |
| ~~REDIRECT>research:vslam:lsdslam~~ |
| |
====== LSD-SLAM: Large-Scale Direct Monocular SLAM ====== | |
**Contact:** [[members:engelj]], [[schoepst@in.tum.de|Thomas Schöps]], [[members:cremers]] | |
| |
**LSD-SLAM** is a novel, direct monocular SLAM technique: Instead of using keypoints, it directly operates on image intensities both for tracking and mapping. The camera is tracked using **direct image alignment**, while geometry is estimated in the form of **semi-dense depth maps**, obtained by **filtering** over many pixelwise stereo comparisons. We then build a **Sim(3) pose-graph of keyframes**, which allows to build scale-drift corrected, large-scale maps including loop-closures. | |
LSD-SLAM runs in **real-time on a CPU**, and even on a modern smartphone. | |
<html><center><span style="color:red;font-size:15pt"><b>Code Available (see below)!</b></span></center></html> | |
| |
<html><center><iframe width="640" height="360" src="//www.youtube.com/embed/GnuQzP3gty4" frameborder="0" allowfullscreen></iframe></center></html> | |
| |
<html><br><br><h1 class="sectionedit1">Difference to keypoint-based methods</h1></html> | |
<html><center></html>{{:research:lsdslam:directvskp.png?500|}}<html></center><br></html> | |
As direct method, LSD-SLAM uses all information in the image, including e.g. edges -- while keypoint-based approaches can only use small patches around corners. | |
This leads to higher accuracy and more robustness in sparsely textured environments (e.g. indoors), and a much denser 3D reconstruction. Further, as the proposed piselwise depth-filters incorporate many small-baseline stereo comparisons instead of only few large-baseline frames, there are much less outliers. | |
| |
<html><br><br><h1 class="sectionedit1">Building a global map</h1></html> | |
[[http://vision.in.tum.de/_media/research/lsdslam/pointcloud.jpg|{{:research:lsdslam:pointcloud.jpg?650|}}]] | |
[[http://vision.in.tum.de/_media/research/lsdslam/depthmaps.jpg|{{:research:lsdslam:depthmaps.jpg?650|}}]] | |
(click on the images for full resolution) | |
| |
LSD-SLAM builds a pose-graph of keyframes, each containing an estimated semi-dense depth map. Using a novel direct image alignment forumlation, we directly track Sim(3)-constraints between keyframes (i.e., rigid body motion + scale), which are used to build a pose-graph which is then optimized. This formulation allows to detect and correct substantial scale-drift after large loop-closures, and to deal with large scale-variation within the same map. | |
| |
<html><br><br><h1 class="sectionedit1">Mobile Implementation</h1></html> | |
The approach even runs on a smartphone, where it can be used for AR. The estimated semi-dense depth maps are in-painted and completed with an estimated ground-plane, which then allows to implement basic physical interaction with the environment. | |
| |
<html><center><iframe width="640" height="360" src="//www.youtube.com/embed/X0hx2vxxTMg" frameborder="0" allowfullscreen></iframe></center></html> | |
| |
| |
<html><br><br><h1 class="sectionedit1">Software</h1></html> | |
LSD-SLAM is on github: | |
[[http://github.com/tum-vision/lsd_slam]] | |
| |
We support only ROS-based build system tested on Ubuntu 12.04 or 14.04 and ROS Indigo or Fuerte. However, ROS is only used for input (video), output (pointcloud & poses) and parameter handling; ROS-dependent code is tightly wrapped and can easily be replaced. To avoid overhead from maintaining different build-systems however, we do not offer an out-of-the-box ROS-free version. Android-specific optimizations and AR integration are not part of the open-source release. | |
| |
Detailled installation and usage instructions can be found in the README.md, including descriptions of the most important parameters. For best results, we recommend using a monochrome [[http://en.ids-imaging.com/store/ui-1221le.html|global-shutter camera]] with [[http://www.lensation.de/images/PDF/BM2420.pdf|fisheye lens]]. | |
| |
If you use our code, please cite our respective publications (see below). We are excited to see what you do with LSD-SLAM, if you want drop us a quick hint if you have nice videos / pictures / models / applications. | |
| |
| |
<html><br><br><h1 class="sectionedit1">Datasets</h1></html> | |
To get you started, we provide some example sequences including the input video and camera calibration, the complete generated pointcloud to be displayed with the ''lsd_slam_viewer'', as well as a (sparsified) pointcloud as .ply, which can be displayed e.g. using meshlab. | |
| |
Hint: Use ''rosbag play -r 25 X_pc.bag'' while the ''lsd_slam_viewer'' is running to replay the result of real-time SLAM at 25x speed, building up the full reconstruction whithin seconds. | |
| |
* **Desk Sequence (0:55min, 640x480 @ 50fps)** | |
* <html><center><iframe width="640" height="360" src="//www.youtube.com/embed/UacKN2WDLCg" frameborder="0" allowfullscreen></iframe></center></html> | |
* Video: [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_room.bag.zip|[.bag]]] [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_room_images.zip|[.png]]] | |
* Pointcloud: [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_room_pc.bag.zip|[.bag]]] [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_room_pc.ply|[.ply]]] | |
| |
* **Machine Sequence (2:20min, 640x480 @ 50fps)** | |
* <html><center><iframe width="640" height="360" src="//www.youtube.com/embed/6KRlwqubLIU" frameborder="0" allowfullscreen></iframe></center></html> | |
* Download Video: [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_machine.bag.zip|[.bag]]] [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_machine_images.zip|[.png]]] | |
* Download Pointcloud: [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_machine_pc.bag.zip|[.bag]]] [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_machine_pc.ply|[.ply]]] | |
| |
* **Foodcourt Sequence (12min, 640x480 @ 50fps)** | |
* <html><center><iframe width="640" height="360" src="//www.youtube.com/embed/aBVXfqumTXc" frameborder="0" allowfullscreen></iframe></center></html> | |
* Download Video: [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_foodcourt.bag.zip|[.bag]]] [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_foodcourt_images.zip|[.png]]] | |
* Download Pointcloud: [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_foodcourt_pc.bag.zip|[.bag]]] [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_foodcourt_pc.ply|[.ply]]] | |
| |
* **ECCV Sequence (7:00min, 640x480 @ 50fps)** | |
* <html><center><iframe width="640" height="360" src="//www.youtube.com/embed/isHXcv_AeFg" frameborder="0" allowfullscreen></iframe></center></html> | |
* **Enable FabMap for large loop-closures for this sequence!** | |
* Video: [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_eccv.bag.zip|[.bag]]] [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_eccv_images.zip|[.png]]] | |
* Pointcloud: [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_eccv_pc.bag.zip|[.bag]]] [[http://vmcremers8.informatik.tu-muenchen.de/lsd/LSD_eccv_pc.ply|[.ply]]] | |
| |
| |
| |
<html><br><br><h1 class="sectionedit1">License</h1></html> | |
LSD-SLAM is released under the GPLv3 license. A professional version under a different licensing agreement intended for commercial use is available [[http://www.fablitec.com/products/lsd-slam/|here]]. Please contact us if you are interested. | |
| |
| |
| |
====== Related publications ====== | |
<bibtex> | |
<keywords>semidense</keywords> | |
</bibtex> | |