Login
This is an old revision of the document!
Image-based 3D Reconstruction
Contact: Martin Oswald, Maria Klodt
For a human, it is usually an easy task to get an idea of the 3D structure shown in an image. Due to the loss of one dimension in the projection process, the estimation of the true 3D geometry is difficult and a so called ill-posed problem, because usually infinitely many different 3D surfaces may produce the same set of images.
3D reconstruction of barley from 25 input images.
3D Reconstruction from multiple views
The goal of multiview 3D reconstruction is to infer geometrical structure of a scene captured by a collection of images. Usually the camera position and internal parameters are assumed to be known or they can be estimated from the set of images. By using multiple images, 3D information can be (partially) recovered by solving a pixel-wise correspondence problem. Since automatic correspondence estimation is usually ambiguous and incomplete further knowledge (prior knowledge) about the object is necessary. A typical prior is assume that the object surface is smooth.
Our research is focused on convex variational methods. The 3D reconstruction problem is formulated as an energy minimization problem.
Due to the convexity of this energy, any (local) minimizer corresponds to the global minimum of this energy.
Minimizers of this energy are found with iterative numerical optimization methods which evolve the surface gradually from the initial surface to best one with respect to energy functional.
As a further consequence of the convexity, these methods are independent of the initialization. The initial surface can be of any shape, for example a simple box.
Two example reconstructions are shown below along with some of their corresponding input images.
Related publications
Export as PDF, XML, TEX or BIB
Book Chapters | Journal Articles | Conference and Workshop Papers
Book Chapters
2011
[] 
Convex Relaxation Techniques for Segmentation, Stereo and Multiview Reconstruction , Chapter in Markov Random Fields for Vision and Image Processing, MIT Press, 2011.
Book Chapters | Journal Articles | Conference and Workshop Papers
Journal Articles
2020
[] 
Photometric Depth Super-Resolution , In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), volume 42, 2020. ([supp] [project page])
2019
[] 
A Non-invasive 3D Body Scanner and Software Tool towards Analysis of Scoliosis , In BioMed Research International (BMRI), 2019. ([pdf])
2012
[] 
Fast Joint Estimation of Silhouettes and Dense 3D Geometry from Multiple Images , In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 34, 2012.
2011
[] 
A Variational Approach to Vesicle Membrane Reconstruction from Fluorescence Imaging , In Pattern Recognition, volume 44, 2011.
[] 
Multiview Stereo and Silhouette Consistency via Convex Functionals over Convex Domains , In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 33, 2011.
2009
[] 
Continuous Global Optimization in Multiview 3D Reconstruction , In International Journal of Computer Vision, volume 84, 2009.
2007
[] 
Weighted Minimal Hypersurface Reconstruction , In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 29, 2007.
Book Chapters | Journal Articles | Conference and Workshop Papers
Conference and Workshop Papers
2023
[] 
High-Quality RGB-D Reconstruction via Multi-View Uncalibrated Photometric Stereo and Gradient-SDF , In IEEE Winter Conference on Applications of Computer Vision (WACV), 2023. ([code] [project page])
Spotlight Presentation
2020
[] 
Inferring Super-Resolution Depth from a Moving Light-Source Enhanced RGB-D Sensor: A Variational Approach , In IEEE Winter Conference on Applications of Computer Vision (WACV), 2020. ([poster] [presentation] [code] [cvf])
Spotlight Presentation
2019
[] 
Combining Depth Fusion and Photometric Stereo for Fine-Detailed 3D Models , In Scandinavian Conference on Image Analysis (SCIA), 2019. ([slides] [poster])
Oral Presentation, received the SCIA 2019 Honourable Mention award
2018
[] 
Fight ill-posedness with ill-posedness: Single-shot variational depth super-resolution from shading , In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018. ([supp] [poster] [slides] [code] [cvf] [video])
Spotlight Presentation
2017
[] 
Depth Super-Resolution Meets Uncalibrated Photometric Stereo , In IEEE International Conference on Computer Vision Workshops (ICCVW), 2017. ([code][slides] [cvf])
Oral Presentation at ICCV Workshop on Color and Photometry in Computer Vision [] 
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting , In International Conference on Computer Vision (ICCV), 2017. ([slides] [poster] [dataset] [code])
[] 
Efficient Online Surface Correction for Real-time Large-Scale 3D Reconstruction , In British Machine Vision Conference (BMVC), 2017. ([poster] [supplementary])
[] 
De-noising, Stabilizing and Completing 3D Reconstructions On-the-go using Plane Priors , In International Conference on Robotics and Automation (ICRA), 2017. ([video])
2014
[] 
Generalized Connectivity Constraints for Spatio-temporal 3D Reconstruction , In European Conference on Computer Vision (ECCV), 2014.
[] 
Surface Normal Integration for Convex Space-time Multi-view Reconstruction , In British Machine Vision Conference (BMVC), 2014.
[] 
Spatial and Temporal Interpolation of Multi-View Image Sequences , In German Conference on Pattern Recognition (GCPR), volume 36, 2014.
[] 
Volumetric 3D Mapping in Real-Time on a CPU , In International Conference on Robotics and Automation (ICRA), 2014.
2013
[] 
Large-Scale Multi-Resolution Surface Reconstruction from RGB-D Sequences , In IEEE International Conference on Computer Vision (ICCV), 2013.
[] 
A Convex Relaxation Approach to Space Time Multi-view 3D Reconstruction , In ICCV Workshop on Dynamic Shape Capture and Analysis (4DMOD), 2013.
[] 
Fast and Accurate Large-scale Stereo Reconstruction using Variational Methods , In ICCV Workshop on Big Data in 3D Computer Vision, 2013.
2011
[] 
Decoupling Photometry and Geometry in Dense Variational Camera Calibration , In IEEE International Conference on Computer Vision (ICCV), 2011.
2010
[] 
Parallel Generalized Thresholding Scheme for Live Dense Geometry from a Handheld Camera , In ECCV Workshop on Computer Vision on GPUs (CVGPU), 2010.
[] 
Real-Time Dense Geometry from a Handheld Camera , In Pattern Recognition (Proc. DAGM), 2010.
[] 
Anisotropic Minimal Surfaces Integrating Photoconsistency and Normal Information for Multiview Stereo , In European Conference on Computer Vision (ECCV), 2010.
2009
[] 
A Superresolution Framework for High-Accuracy Multiview Reconstruction , In Pattern Recognition (Proc. DAGM), 2009.
Received DAGM Best Paper Award [] 
Continuous Ratio Optimization via Convex Relaxation with Applications to Multiview 3D Reconstruction , In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009.
2008
[] 
An Experimental Comparison of Discrete and Continuous Shape Optimization Methods , In European Conference on Computer Vision (ECCV), 2008.
[] 
Integration of Multiview Stereo and Silhouettes via Convex Functionals on Convex Domains , In European Conference on Computer Vision (ECCV), 2008.
2007
[] 
Continuous Global Optimization in Multiview 3D Reconstruction , In Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR), Springer, volume 4679, 2007.
[] 
Propagated Photoconsistency and Convexity in Variational Multiview 3D Reconstruction , In Workshop on Photometric Analysis for Computer Vision, 2007.
2006
[] 
Robust variational segmentation of 3D objects from multiple views , In Pattern Recognition (Proc. DAGM) (K. Fet al., ed.), Springer, volume 4174, 2006.
2005
[] 
Reconstructing the Geometry of Flowing Water , In IEEE International Conference on Computer Vision (ICCV), IEEE, 2005.
[] 
Spacetime-Continous Geometry Meshes from Multi-View Video Sequences , In Proceedings of the IEEE International Conference on Image Processing, IEEE Computer Society, 2005.
2004
[] 
Space-Time Isosurface Evolution for Temporally Coherent 3D Reconstruction , In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, volume I, 2004.
[] 
Weighted Minimal Hypersurfaces and Their Applications in Computer Vision , In European Conference on Computer Vision (ECCV), Springer, volume 3022, 2004.
[] 
Spacetime-coherent Geometry Reconstruction from Multiple Video Streams , In 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT) (Y. Aloimonos, ed.), IEEE, 2004.
2003
[] 
Joint 3D Reconstruction and Background Separation in Multiple Views using Graph Cuts , In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, volume I, 2003.
