Robotic 3D Vision (3h +1h, 5ECTS)
WS 2017/18, TU München
News
20.03.2018: The retake exam will be conducted as oral exams (30min duration each). Only students that have registered for the exam in TUMOnline can take the exam. Possible time slots are on March 22nd and March 23rd 2018. Students who have registered for the exam in TUMOnline should have received an email with further instructions on the assignment of exam dates. If you are registered for the exam in TUMOnline but have not received an email, please contact us at rob3dvis-ws17@vision.in.tum.de
07.03.2018: The retake exam will be conducted either as oral exams (30min duration each) on 22./23.03.2018 or as a written exam on 26.03.2018, 13:00-14:30. The type of exam will be decided and announced by the course organizers after the registration period on 20.03.2018. Only students that have registered for the retake exam in TUMOnline during the registration period (05.03.2018-19.03.2018) can take the exam.
07.02.2018: Exercise sheet 6 has been updated and is available on the course material page.
02.02.2018: Students who have registered for the exam in TUMOnline should have received an email with the assignment of oral exam time slots. The assignments can also be viewed here. If you are registered for the exam in TUMOnline but have not been assigned to an oral exam date, please contact us at rob3dvis-ws17@vision.in.tum.de
30.01.2018: Exercise sheet 6 is available on the course material page. Due date is Wednesday, 07.02.2018, 23:59
26.01.2018: The exam will be conducted as oral exams (30min duration each). Only students that have registered for the exam in TUMOnline can take the exam. Possible time slots are on Feb 20th and Feb 21st 2018. Students who have registered for the exam in TUMOnline should have received an email with further instructions on the assignment of exam dates. If you are registered for the exam in TUMOnline but have not received an email, please contact us at rob3dvis-ws17@vision.in.tum.de
24.01.2018: Updated material for exercise sheet 5 (surfel_data.txt) is available on the course material page.
16.01.2018: Exercise sheet 5 is available on the course material page. Due date is Wednesday, 24.01.2018, 23:59
19.12.2017: Exercise sheet 4 is available on the course material page. Due date is Wednesday, 10.01.2018, 23:59
06.12.2017: The date for the exam (written, tentative) is 21.02.2018, 10:30-12:00. Registration for the exam is possible in TUMOnline and is open until Jan. 15th, 2018. Note that we also might decide to conduct the exam as oral exams (tbd) depending on the number of registrations.
05.12.2017: Exercise sheet 3 is available on the course material page. Due date is Wednesday, 13.12.2017, 23:59
21.11.2017: Exercise sheet 2 is available on the course material page. Due date is Wednesday, 29.11.2017, 23:59
07.11.2017: Exercise sheet 1 is available on the course material page. Due date is Wednesday, 15.11.2017, 23:59
02.11.2017: Exercises in the future will be held in Room 02.05.014.
Lecture
Time and Date:
Tuesdays (every week), 14.15h - 15.45h in room 00.09.038 (starting from Oct 24th)
Thursdays (every second week, alternating with exercise), 14.15h - 15.45h in room 00.11.038 (starting from Oct 19th)
Lecturer: Prof. Dr. Jörg Stückler
The lectures are held in English.
Exercises
Location: Room 02.05.014
Time and Date: Thursdays (every second week, alternating with lecture), 14.15h - 16.00h
Organization/Tutor: Rui Wang
Questions: rob3dvis-ws17@vision.in.tum.de
TUMOnline course entries:
https://campus.tum.de/tumonline/wbLv.wbShowLVDetail?pStpSpNr=950348493
https://campus.tum.de/tumonline/wbLv.wbShowLVDetail?pStpSpNr=950348496
Study programme: M. Sc. Informatics
Exam date: The exam will be conducted as oral exams (30min duration each). Only students that have registered for the exam in TUMOnline can take the exam. Possible time slots are on Feb 20th and Feb 21st 2018. Students who have registered for the exam in TUMOnline should have received an email with further instructions on the assignment of exam dates.
Registration for the exam is only possible in TUMOnline and is open until Jan. 15th, 2018.
Summary
This lecture will cover vision-based approaches to 3D robotic perception such as 3D simultaneous localization and mapping or 3D object detection with monocular, stereo and RGB-D cameras. Vision-based approaches attract significant attention in research and industrial applications due to the cost-effectiveness of passive cameras but also RGB-D cameras in comparison to sensors such as lasers or radars. The course will treat the following topics:
- Introduction to robotic 3D vision
- Multi-view geometry and non-linear least squares optimization basics
- Indirect and direct methods for rigid image alignment
- Visual odometry, indirect and direct methods
- Visual-inertial odometry
- Visual SLAM, pose graph optimization, indirect and direct bundle adjustment
- 3D map representations
- 3D object detection
- 3D object tracking
- Non-rigid registration methods and SLAM, DynamicFusion
Prerequisites
Passion for mathematics to solve complex computer vision problems and a solid background in basic mathematics such as linear algebra and analysis are required. A solid background in basics of multi-view geometry (f.e. acquired through successful participation in the lecture “Computer Vision II: Multi-View Geometry” (IN2228)) is recommended. The course will include exercises for which previous knowledge in programming with Python and Matlab is required.
Detailed Schedule (tentative)
- Thu 19.10.17, Lecture 1: Introduction
- Tue 24.10.17, Lecture 2: Image formation, multi-view geometry
- Thu 26.10.17, Lecture 3: Probabilistic state estimation - filtering
- Tue 31.10.17, no lecture (public holiday, Reformationstag)
- Thu 02.11.17, Exercise 1: Matlab
- Tue 07.11.17, Lecture 4: Probabilistic state estimation - full posterior optimization
- Thu 09.11.17, Lecture 5: Visual odometry 1 - introduction, indirect methods
- Tue 14.11.17, Lecture 6: Visual odometry 2 - indirect methods cont.
- Thu 16.11.17, Exercise 2: Image formation, Extended Kalman Filter
- Tue 21.11.17, Lecture 7: Keypoints
- Thu 23.11.17, Lecture 8: Visual odometry 3 - direct methods
- Tue 28.11.17, Lecture 9: Visual-inertial odometry
- Thu 30.11.17, Exercise 3: Graphical models, indirect visual odometry
- Tue 05.12.17, Lecture 10: Visual SLAM 1 - introduction, bundle adjustment
- Thu 07.12.17, Lecture 11: Visual SLAM 2 - online SLAM, indirect EKF-SLAM
- Tue 12.12.17, Lecture 12: Visual SLAM 3 - tracking and mapping, pose graph optimization, place recognition
- Thu 14.12.17, Exercise 4: Keypoint detection and matching, direct visual odometry
- Tue 19.12.17, Lecture 13: Visual SLAM 4 - direct methods
- Thu 21.12.17, Lecture 14: Visual SLAM 5 - DSO, SLAM overview
- Tue 09.01.18, Lecture 15: 3D Object Detection 1
- Thu 11.01.18, Exercise 5: EKF-SLAM, direct visual SLAM, pose graph optimization
- Tue 16.01.18, Lecture 16: 3D Object Detection 2
- Thu 18.01.18, Lecture 17: 3D Object Tracking
- Tue 23.01.18, Lecture 18: Stereo and Dense Reconstruction
- Thu 25.01.18, Exercise 6: Surfel-Pair Matching, ICP Algorithm
- Tue 30.01.18, Lecture 19: Selected Research Topics: Stereo DSO and Online Photometric Calibration (Rui Wang)
- Thu 01.02.18, Lecture 20: Map Representations
- Tue 06.02.18, Lecture 21: Summary / Repetition
- Thu 08.02.18, Exercise 7: Dense Stereo Reconstruction and Map Representations
Lecture Material
Slides and exercise sheets can be accessed here.