This is an old revision of the document!
Photometric Depth Super-Resolution Dataset
Photometric Depth Super-Resolution
Bjoern Haefner1\ \ \ \ Songyou Peng2\ \ \ \ Alok Verma1\ \ \ \ Yvain Quéau3\ \ \ \ Daniel Cremers1
1Technical University of Munich\ \ \ \ 2University of Illinois
at Urbana-Champaign\ \ \ \ 3University of Caen
Submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) Special Issue on RGB-D Vision: Methods and Applications
This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and real-world scenarios.
Code
The code that generated the data shown here is available on github:
https://github.com/BjoernHaefner/DepthSRfromShading
https://github.com/pengsongyou/SRmeetsPS.
Dataset
The following dataset contains RGB-D sequences and reconstructed 3D models of multiple different scenes. We captured the RGB-D data under different scaling factors using an Asus Xtion Pro and an Intel RealSense D415 RGB-D sensor. Please refer to the respective publication when using this data.
Format
For each scene of the Photometric Depth Super-Resolution dataset, we provide the respective RGB-D sequence as well as the refined 3D models. Each RGB-D sequence contains:
- Color frames
- Depth frames
- Masks
- Intrinsic parameters (default factory calibration).
Rucksack 1
Download (mat-files): rucksack1_sfs.zip | rucksack1_ups.zip
Download (png-file of deep net albedo estimate): albedo_deep.png
Download (obj-files): rucksack1_input_mesh.zip | rucksack1_sfs_mesh.zip | rucksack1_deep_mesh.zip | rucksack1_ups_mesh.zip
Rucksack 2
Android
Basecap
Minion
Blanket
Clothes
Monkey
Wool
Face 1 Expression 1
Face 1 Expression 2
Face 2 Expression 1
Face 2 Expression 2
Face 3
Face 4
Tabletcase
Shirt
Bag
Backpack
Oven mitt
Hat
Vase
License
Unless stated otherwise, all data in the Intrinsic3D Dataset is licensed under a Creative Commons 4.0 Attribution License (CC BY 4.0).
Related Publications
Export as PDF, XML, TEX or BIB
Journal Articles
2020
[] Photometric Depth Super-Resolution , In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), volume 42, 2020. ([supp] [project page])
Conference and Workshop Papers
2018
[] Fight ill-posedness with ill-posedness: Single-shot variational depth super-resolution from shading , In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018. ([supp] [poster] [slides] [code] [cvf] [video])
Spotlight Presentation
2017
[] Depth Super-Resolution Meets Uncalibrated Photometric Stereo , In IEEE International Conference on Computer Vision Workshops (ICCVW), 2017. ([code][slides] [cvf])
Oral Presentation at ICCV Workshop on Color and Photometry in Computer Vision