Visual SLAM
In Simultaneous Localization And Mapping, we track the pose of the sensor while creating a map of the environment. In particular, our group has a strong focus on direct methods, where, contrary to the classical pipeline of feature extraction and matching, we directly optimize intensity errors.
Our experience includes various sensor modalities, such as monocular, stereo and RGB-D cameras, but also visual-inertial setups. Besides our research for new methods, we provide public datasets for evaluation.
If you are interested in the wider area of SLAM, chances are that our interests overlap with yours. We would be happy to hear from you!
Contact
Related publications
Export as PDF, XML, TEX or BIB
2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2024
Conference and Workshop Papers
[] Physically-Based Photometric Bundle Adjustment in Non-Lambertian Environments , In International Conference on Intelligent Robots and Systems (IROS), 2024.
2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2023
Conference and Workshop Papers
[] Learning Correspondence Uncertainty via Differentiable Nonlinear Least Squares , In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. ([project page])
[] Behind the Scenes: Density Fields for Single View Reconstruction , In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023. ([project page])
[] Power Bundle Adjustment for Large-Scale 3D Reconstruction , In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023. ([project page][video])
2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2022
Journal Articles
[] DM-VIO: Delayed Marginalization Visual-Inertial Odometry , In IEEE Robotics and Automation Letters (RA-L) & International Conference on Robotics and Automation (ICRA), volume 7, 2022. ([arXiv][video][project page][supplementary][code])
Conference and Workshop Papers
[] DirectTracker: 3D Multi-Object Tracking Using Direct Image Alignment and Photometric Bundle Adjustment , In International Conference on Intelligent Robots and Systems (IROS), 2022. ([project page])
[] The Probabilistic Normal Epipolar Constraint for Frame-To-Frame Rotation Optimization under Uncertain Feature Positions , In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. ([project page])
2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2021
Preprints
[] Event-Based Feature Tracking in Continuous Time with Sliding Window Optimization , In arXiv preprint, 2021.
Conference and Workshop Papers
[] Multidirectional Conjugate Gradients for Scalable Bundle Adjustment , In German Conference on Pattern Recognition (GCPR), 2021. ([presentation])
[] TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo , In Conference on Robot Learning (CoRL), 2021. ([GitHub][video][project page])
3DV'21 Best Demo Award [] TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset , In International Conference on Intelligent Robots and Systems (IROS), 2021. ([project page])
[] Towards Robust Monocular Visual Odometry for Flying Robots on Planetary Missions , In International Conference on Intelligent Robots and Systems (IROS), 2021. ([code][video])
[] Square Root Marginalization for Sliding-Window Bundle Adjustment , In IEEE International Conference on Computer Vision (ICCV), 2021. ([project page])
[] Square Root Bundle Adjustment for Large-Scale Reconstruction , In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. ([project page])
[] Tight Integration of Feature-based Relocalization in Monocular Direct Visual Odometry , In Proc. of the IEEE International Conference on Robotics and Automation (ICRA), 2021. ([project page])
[] MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera , In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. ([project page])
2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2020
Book Chapters
[] TUM Flyers: Vision—Based MAV Navigation for Systematic Inspection of Structures , Chapter in Bringing Innovative Robotic Technologies from Research Labs to Industrial End-users (F. Caccavale, C. Ott, B. Winkler, Z. Taylor, eds.), Springer International Publishing, 2020.
[] RGB-D Vision , Chapter in Encyclopedia of Robotics (M.H. Ang, O. Khatib, B. Siciliano, eds.), Springer Berlin Heidelberg, 2020.
Journal Articles
[] GN-Net: The Gauss-Newton Loss for Multi-Weather Relocalization , In IEEE Robotics and Automation Letters (RA-L), volume 5, 2020. ([arXiv][video][project page][supplementary])
[] Visual-Inertial Mapping with Non-Linear Factor Recovery , In IEEE Robotics and Automation Letters (RA-L) & Int. Conference on Intelligent Robotics and Automation (ICRA), IEEE, volume 5, 2020. ([arxiv])
Conference and Workshop Papers
[] LM-Reloc: Levenberg-Marquardt Based Direct Visual Relocalization , In International Conference on 3D Vision (3DV), 2020. ([arXiv][project page][video][supplementary][poster])
[] Distributed Photometric Bundle Adjustment , In International Conference on 3D Vision (3DV), 2020. ([project page][code])
Oral Presentation [] 4Seasons: A Cross-Season Dataset for Multi-Weather SLAM in Autonomous Driving , In Proceedings of the German Conference on Pattern Recognition (GCPR), 2020. ([project page][arXiv][video])
[] DH3D: Deep Hierarchical 3D Descriptors for Robust Large-Scale 6DoF Relocalization , In European Conference on Computer Vision (ECCV), 2020. ([project page][code][supplementary][arxiv])
Spotlight Presentation [] D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry , In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Oral Presentation [] Efficient Derivative Computation for Cumulative B-Splines on Lie Groups , In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Oral Presentation [] DirectShape: Photometric Alignment of Shape Priors for Visual Vehicle Pose and Shape Estimation , In Proc. of the IEEE International Conference on Robotics and Automation (ICRA), 2020. ([video][presentation][project page][supplementary][arxiv])
2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2019
Book Chapters
[] A Review and Quantitative Evaluation of Direct Visual–Inertial Odometry , Chapter in Multimodal Scene Understanding (M. Yang, B. Rosenhahn, V. Murino, eds.), Academic Press, 2019.
Conference and Workshop Papers
[] Visual-inertial sensor fusion with a bio-inspired polarization compass for navigation of MAVs , In 11th International Micro Air Vehicle Competition and Conference (IMAV), 2019.
[] Rolling-Shutter Modelling for Visual-Inertial Odometry , In International Conference on Intelligent Robots and Systems (IROS), 2019. ([arxiv])
2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2018
Journal Articles
[] Omnidirectional DSO: Direct Sparse Odometry with Fisheye Cameras , In IEEE Robotics and Automation Letters & Int. Conference on Intelligent Robots and Systems (IROS), IEEE, 2018. ([arxiv])
[] Online Photometric Calibration of Auto Exposure Video for Realtime Visual Odometry and SLAM , In IEEE Robotics and Automation Letters (RA-L), volume 3, 2018. (This paper was also selected by ICRA'18 for presentation at the conference.[arxiv][video][code][project])
ICRA'18 Best Vision Paper Award - Finalist [] Challenges in Monocular Visual Odometry: Photometric Calibration, Motion Bias and Rolling Shutter Effect , In In IEEE Robotics and Automation Letters (RA-L) & Int. Conference on Intelligent Robots and Systems (IROS), volume 3, 2018. ([arxiv])
[] Direct Sparse Odometry , In IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018.
Conference and Workshop Papers
[] The Double Sphere Camera Model , In Proc. of the Int. Conference on 3D Vision (3DV), 2018. ([arxiv])
[] Direct Sparse Odometry With Rolling Shutter , In European Conference on Computer Vision (ECCV), 2018. ([supplementary][arxiv])
Oral Presentation [] Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry , In European Conference on Computer Vision (ECCV), 2018. ([arxiv],[supplementary],[project])
Oral Presentation [] LDSO: Direct Sparse Odometry with Loop Closure , In International Conference on Intelligent Robots and Systems (IROS), 2018. ([arxiv][video][code][project])
[] The TUM VI Benchmark for Evaluating Visual-Inertial Odometry , In International Conference on Intelligent Robots and Systems (IROS), 2018. ([arxiv])
[] Direct Sparse Visual-Inertial Odometry using Dynamic Marginalization , In International Conference on Robotics and Automation (ICRA), 2018. ([supplementary][video][arxiv])
[] StaticFusion: Background Reconstruction for Dense RGB-D SLAM in Dynamic Environments , In 2018 IEEE International Conference on Robotics and Automation, ICRA 2018, Brisbane, Australia, May 21-25, 2018, IEEE, 2018.
[] Joint Representation of Primitive and Non-primitive Objects for 3D Vision , In 2018 International Conference on 3D Vision, 3DV 2018, Verona, Italy, September 5-8, 2018, IEEE Computer Society, 2018.
2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2017
Conference and Workshop Papers
[] Keyframe-Based Visual-Inertial Online SLAM with Relocalization , In IEEE/RSJ Int. Conference on Intelligent Robots and Systems, IROS, 2017.
[] Efficient Online Surface Correction for Real-time Large-Scale 3D Reconstruction , In British Machine Vision Conference (BMVC), 2017. ([poster] [supplementary])
[] From Monocular SLAM to Autonomous Drone Exploration , In European Conference on Mobile Robots (ECMR), 2017. ([arXiv])
[] De-noising, Stabilizing and Completing 3D Reconstructions On-the-go using Plane Priors , In International Conference on Robotics and Automation (ICRA), 2017. ([video])
[] Real-time variational stereo reconstruction with applications to large-scale dense SLAM , In IEEE Intelligent Vehicles Symposium, IV 2017, Los Angeles, CA, USA, June 11-14, 2017, IEEE, 2017.
2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2016
Conference and Workshop Papers
[] Scaling the world of monocular SLAM with INS-measurements for UAS navigation , In 19th International Conference on Information Fusion, FUSION 2016, Heidelberg, Germany, July 5-8, 2016, IEEE, 2016.
[] Direct Sparse Odometry , In arXiv:1607.02565, 2016.
[] A Photometrically Calibrated Benchmark For Monocular Visual Odometry , In arXiv:1607.02555, 2016.
[] CPA-SLAM: Consistent Plane-Model Alignment for Direct RGB-D SLAM , In International Conference on Robotics and Automation (ICRA), 2016.
[] Direct Visual-Inertial Odometry with Stereo Cameras , In International Conference on Robotics and Automation (ICRA), 2016.
2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2015
Journal Articles
[] Multi-Layered Mapping and Navigation for Autonomous Micro Aerial Vehicles , In Journal of Field Robotics, 2015. (to appear)
[] Fast Visual Odometry for 3-D Range Sensors , In IEEE Transactions on Robotics, volume 31, 2015. ([video])
Conference and Workshop Papers
[] Dense Continuous-Time Tracking and Mapping with Rolling Shutter RGB-D Cameras , In IEEE International Conference on Computer Vision (ICCV), 2015. ([video][supplementary][datasets])
[] Reconstructing Street-Scenes in Real-Time From a Driving Car , In Proc. of the Int. Conference on 3D Vision (3DV), 2015.
[] Large-Scale Direct SLAM for Omnidirectional Cameras , In International Conference on Intelligent Robots and Systems (IROS), 2015.
[] Large-Scale Direct SLAM with Stereo Cameras , In International Conference on Intelligent Robots and Systems (IROS), 2015.
2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2014
Journal Articles
[] Multi-Resolution Surfel Maps for Efficient Dense 3D Modeling and Tracking , In Journal of Visual Communication and Image Representation, volume 25, 2014.
[] Scale-Aware Navigation of a Low-Cost Quadrocopter with a Monocular Camera , In Robotics and Autonomous Systems (RAS), volume 62, 2014.
Conference and Workshop Papers
[] Local multi-resolution representation for 6D motion estimation and mapping with a continuously rotating 3D laser scanner , In Proc. of the IEEE Int. Conf. on Robotics and Automation (ICRA), 2014.
[] Combining the Strengths of Sparse Interest Point and Dense Image Registration for RGB-D Odometry , In Proc. of the Joint 45th International Symposium on Robotics (ISR) and 8th German Conference on Robotics (ROBOTIK), 2014.
[] Local Multi-Resolution Surfel Grids for MAV Motion Estimation and 3D Mapping , In Proc. of the 13th International Conference on Intelligent Autonomous Systems (IAS), 2014.
[] Visual-Inertial Navigation for a Camera-Equipped 25g Nano-Quadrotor , In IROS2014 Aerial Open Source Robotics Workshop, 2014.
[] Submap-based Bundle Adjustment for 3D Reconstruction from RGB-D Data , In German Conference on Pattern Recognition (GCPR), 2014. ([slides])
Oral Presentation [] Semi-Dense Visual Odometry for AR on a Smartphone , In International Symposium on Mixed and Augmented Reality, 2014.
Best Short Paper Award [] LSD-SLAM: Large-Scale Direct Monocular SLAM , In European Conference on Computer Vision (ECCV), 2014.
Oral Presentation [] Collision Avoidance for Quadrotors with a Monocular Camera , In Proc. of The 12th International Symposium on Experimental Robotics (ISER), 2014.
[] Volumetric 3D Mapping in Real-Time on a CPU , In International Conference on Robotics and Automation (ICRA), 2014.
[] Event-based 3D SLAM with a depth-augmented dynamic vision sensor , In International Conference on Robotics and Automation (ICRA), 2014.
Other Publications
[] Visual Inertial Control of a Nano-Quadrotor , Master's thesis, Technical University Munich, 2014.
[] Monocular Visual Inertial Odometry on a Mobile Device , Master's thesis, Technical University Munich, 2014.
[] Semi-dense visual SLAM on mobile devices , Master's thesis, Technical University Munich, 2014.
2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2013
Journal Articles
[] 3D Mapping with an RGB-D Camera , In IEEE Transactions on Robotics (T-RO), volume 30, 2013.
Conference and Workshop Papers
[] Hierarchical Object Discovery and Dense Modelling From Motion Cues in RGB-D Video , In Proc. of the 23rd International Joint Conference on Artificial Intelligence (IJCAI), IJCAI/AAAI, 2013.
[] Joint detection and pose tracking of multi-resolution surfel models in RGB-D , In Proc. of the European Conference on Mobile Robots (ECMR), IEEE, 2013.
[] Multi-resolution surfel mapping and real-time pose tracking using a continuously rotating 2D laser scanner , In Proc. of the IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), 2013.
[] Large-Scale Multi-Resolution Surface Reconstruction from RGB-D Sequences , In IEEE International Conference on Computer Vision (ICCV), 2013.
[] Semi-Dense Visual Odometry for a Monocular Camera , In IEEE International Conference on Computer Vision (ICCV), 2013.
[] Dense Tracking and Mapping with a Quadrocopter , In Unmanned Aerial Vehicle in Geomatics (UAV-g), 2013.
[] Dense Visual SLAM for RGB-D Cameras , In Proc. of the Int. Conf. on Intelligent Robot Systems (IROS), 2013.
[] Direct Camera Pose Tracking and Mapping With Signed Distance Functions , In Demo Track of the RGB-D Workshop on Advanced Reasoning with Depth Cameras at the Robotics: Science and Systems Conference (RSS), 2013.
[] Real-Time Camera Tracking and 3D Reconstruction Using Signed Distance Functions , In Robotics: Science and Systems Conference (RSS), 2013.
[] Robust Odometry Estimation for RGB-D Cameras , In International Conference on Robotics and Automation (ICRA), 2013.
Best Vision Paper Award - Finalist
Other Publications
[] Out-of-Core Bundle Adjustment for 3D Workpiece Reconstruction , Master's thesis, Technische Universität München, 2013.
[] Lecture Notes: Visual Navigation for Flying Robots , Technische Universität München, Germany, 2013.
Distinguished with the TUM TeachInf Award for the best lecture in summer term 2013 2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2012
Conference and Workshop Papers
[] Robust Real-Time Registration of RGB-D Images using Multi-Resolution Surfel Representations , In Proc. of ROBOTIK, VDE-Verlag, 2012.
[] Integrating depth and color cues for dense multi-resolution scene mapping using RGB-D cameras , In Proc. of the IEEE Int. Conf. on Multisensor Fusion and Integration for Intelligent Systems (MFI), 2012.
[] Evaluating Egomotion and Structure-from-Motion Approaches Using the TUM RGB-D Benchmark , In Proc. of the Workshop on Color-Depth Camera Fusion in Robotics at the IEEE/RJS International Conference on Intelligent Robot Systems (IROS), 2012.
[] Accurate Figure Flying with a Quadrocopter Using Onboard Visual and Inertial Sensing , In Proc. of the Workshop on Visual Control of Mobile Robots (ViCoMoR) at the IEEE/RJS International Conference on Intelligent Robot Systems (IROS), 2012.
[] A Benchmark for the Evaluation of RGB-D SLAM Systems , In Proc. of the International Conference on Intelligent Robot Systems (IROS), 2012.
[] Camera-Based Navigation of a Low-Cost Quadrocopter , In Proc. of the International Conference on Intelligent Robot Systems (IROS), 2012.
[] An Evaluation of the RGB-D SLAM System , In International Conference on Robotics and Automation (ICRA), 2012.
Other Publications
[] Lecture Notes: Visual Navigation for Flying Robots , Technische Universität München, Germany, 2012.
Distinguished with the TUM TeachInf Award for the best lecture in summer term 2012 [] Odometry from RGB-D Cameras for Autonomous Quadrocopters , Master's thesis, Technical University Munich, 2012.
2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2011
Conference and Workshop Papers
[] Real-Time Visual Odometry from Dense RGB-D Images , In Workshop on Live Dense Reconstruction with Moving Cameras at the Intl. Conf. on Computer Vision (ICCV), 2011.
[] Towards a benchmark for RGB-D SLAM evaluation , In Proc. of the RGB-D Workshop on Advanced Reasoning with Depth Cameras at Robotics: Science and Systems Conf. (RSS), 2011.
[] Real-time 3D visual SLAM with a hand-held camera , In Proc. of the RGB-D Workshop on 3D Perception in Robotics at the European Robotics Forum, 2011.
Other Publications
[] Autonomous Camera-Based Navigation of a Quadrocopter , Master's thesis, Technical University Munich, 2011.
Distinguished with the SIEMENS award for best Master's Thesis 2012 2024 | 2023 | 2022 | 2021 | 2020 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010
2010
Conference and Workshop Papers
[] Improving indoor navigation of autonomous robots by an explicit representation of doors , In Proc. of the IEEE Int. Conf. on Robotics and Automation (ICRA), 2010.
[] Parallel Generalized Thresholding Scheme for Live Dense Geometry from a Handheld Camera , In ECCV Workshop on Computer Vision on GPUs (CVGPU), 2010.
[] Real-Time Dense Geometry from a Handheld Camera , In Pattern Recognition (Proc. DAGM), 2010.