Login
Photometric Depth Super-Resolution Dataset
Photometric Depth Super-Resolution
Bjoern Haefner1 Songyou Peng2 Alok Verma1 Yvain Quéau3 Daniel Cremers1
1Technical University of Munich 2University of Illinois
at Urbana-Champaign 3GREYC, UMR CNRS 6072
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) Special Issue on RGB-D Vision: Methods and Applications
This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and real-world scenarios.
Code
The code that generated the data shown here is available on github:
https://github.com/BjoernHaefner/DepthSRfromShading
https://github.com/pengsongyou/SRmeetsPS.
Dataset
The following dataset contains RGB-D sequences and reconstructed 3D models of multiple different scenes. We captured the RGB-D data under different scaling factors using an Asus Xtion Pro and an Intel RealSense D415 RGB-D sensor. Please refer to the respective publication when using this data.
Format
For each scene of the Photometric Depth Super-Resolution dataset, we provide the respective RGB-D sequence as well as the refined 3D models. Each RGB-D sequence contains:
- Color frames
- Depth frames
- Masks
- Intrinsic parameters (default factory calibration).
Rucksack
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Android
Download (mat-files):
Download (obj-files):
Basecap
Download (mat-files):
Download (obj-files):
Minion
Download (mat-files):
Download (obj-files):
Blanket
Download (mat-files):
Download (obj-files):
Clothes
Download (mat-files):
Download (obj-files):
Monkey
Download (mat-files):
Download (obj-files):
Wool
Download (mat-files):
Download (obj-files):
Face 1
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Face 2
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Face 3
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Face 4
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Face 5
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Face 6
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Tabletcase
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Shirt
Download (mat-files):
Download (obj-files):
Backpack
Download (mat-files):
Download (obj-files):
Ovenmitt
Download (mat-files):
Download (obj-files):
Hat
Download (mat-files):
Download (obj-files):
Vase
Download (mat-files):
Download (obj-files):
License
Unless stated otherwise, all data in the Photometric Depth Super-Resolution Dataset is licensed under a Creative Commons 4.0 Attribution License (CC BY-NC-SA 4.0).
Related Publications
Export as PDF, XML, TEX or BIB
Journal Articles
2020
[] 
Photometric Depth Super-Resolution , In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), volume 42, 2020. ([supp] [project page])
Conference and Workshop Papers
2018
[] 
Fight ill-posedness with ill-posedness: Single-shot variational depth super-resolution from shading , In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018. ([supp] [poster] [slides] [code] [cvf] [video])
Spotlight Presentation
2017
[] 
Depth Super-Resolution Meets Uncalibrated Photometric Stereo , In IEEE International Conference on Computer Vision Workshops (ICCVW), 2017. ([code][slides] [cvf])
Oral Presentation at ICCV Workshop on Color and Photometry in Computer Vision
