Photometric Depth Super-Resolution Dataset
Photometric Depth Super-Resolution
Bjoern Haefner1 Songyou Peng2 Alok Verma1 Yvain Quéau3 Daniel Cremers1
1Technical University of Munich 2University of Illinois
at Urbana-Champaign 3GREYC, UMR CNRS 6072
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/photometricdepthsr_teaser.png?w=1400&tok=ae146c)
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) Special Issue on RGB-D Vision: Methods and Applications
This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and real-world scenarios.
Code
The code that generated the data shown here is available on github:
https://github.com/BjoernHaefner/DepthSRfromShading
https://github.com/pengsongyou/SRmeetsPS.
Dataset
The following dataset contains RGB-D sequences and reconstructed 3D models of multiple different scenes. We captured the RGB-D data under different scaling factors using an Asus Xtion Pro and an Intel RealSense D415 RGB-D sensor. Please refer to the respective publication when using this data.
Format
For each scene of the Photometric Depth Super-Resolution dataset, we provide the respective RGB-D sequence as well as the refined 3D models. Each RGB-D sequence contains:
- Color frames
- Depth frames
- Masks
- Intrinsic parameters (default factory calibration).
Rucksack
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/realsense_rucksack1_teaser.png?w=1400&tok=065028)
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Android
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/realsense_android_teaser.png?w=1400&tok=dbf4d2)
Download (mat-files):
Download (obj-files):
Basecap
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/realsense_basecap_teaser.png?w=1400&tok=e81a90)
Download (mat-files):
Download (obj-files):
Minion
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/realsense_minion_teaser.png?w=1400&tok=9b2201)
Download (mat-files):
Download (obj-files):
Blanket
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/xtion_blanket_teaser.png?w=1400&tok=c25493)
Download (mat-files):
Download (obj-files):
Clothes
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/xtion_clothes_teaser.png?w=1400&tok=4d68c4)
Download (mat-files):
Download (obj-files):
Monkey
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/xtion_monkey_teaser.png?w=1400&tok=f98bfb)
Download (mat-files):
Download (obj-files):
Wool
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/xtion_wool_teaser.png?w=1400&tok=ae773a)
Download (mat-files):
Download (obj-files):
Face 1
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/realsense_face1_teaser.png?w=1400&tok=7b0081)
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Face 2
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/realsense_face2_teaser.png?w=1400&tok=3d2d9b)
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Face 3
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/realsense_face3_teaser.png?w=1400&tok=89d7c9)
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Face 4
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/realsense_face4_teaser.png?w=1400&tok=bb754a)
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Face 5
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/realsense_face5_teaser.png?w=1400&tok=be890d)
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Face 6
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/realsense_face6_teaser.png?w=1400&tok=e595cc)
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Tabletcase
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/xtion_tabletcase_teaser.png?w=1400&tok=95bbbc)
Download (mat-files):
Download (png-file of deep net albedo estimate):
Download (obj-files):
Shirt
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/xtion_shirt_teaser.png?w=1400&tok=4b23a4)
Download (mat-files):
Download (obj-files):
Backpack
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/xtion_backpack_teaser.png?w=1400&tok=482152)
Download (mat-files):
Download (obj-files):
Ovenmitt
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/xtion_ovenmitt_teaser.png?w=1400&tok=7d0f80)
Download (mat-files):
Download (obj-files):
Hat
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/realsense_hat_teaser.png?w=1400&tok=00e848)
Download (mat-files):
Download (obj-files):
Vase
![](https://cvg.cit.tum.de/_media/data/datasets/photometricdepthsr/xtion_vase_teaser.png?w=1400&tok=19924e)
Download (mat-files):
Download (obj-files):
License
Unless stated otherwise, all data in the Photometric Depth Super-Resolution Dataset is licensed under a Creative Commons 4.0 Attribution License (CC BY-NC-SA 4.0).
Related Publications
Export as PDF, XML, TEX or BIB
Journal Articles
2020
[] Photometric Depth Super-Resolution , In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), volume 42, 2020. ([supp] [project page])
Conference and Workshop Papers
2018
[] Fight ill-posedness with ill-posedness: Single-shot variational depth super-resolution from shading , In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018. ([supp] [poster] [slides] [code] [cvf] [video])
Spotlight Presentation
2017
[] Depth Super-Resolution Meets Uncalibrated Photometric Stereo , In IEEE International Conference on Computer Vision Workshops (ICCVW), 2017. ([code][slides] [cvf])
Oral Presentation at ICCV Workshop on Color and Photometry in Computer Vision