Scene Reassembly after Multimodal Digitization

Scene reassembly after multimodal digitization and pipeline evaluation using photorealistic rendering

Jonathan Dyssel Stets^1,†, Alessandro Dal Corso^1,†, Jannik Boll Nielsen¹, Rasmus Ahrenkiel Lyngby¹, Sebastian Hoppe Nesgaard Jensen¹, Jakob Wilm¹, Mads Brix Doest¹, Carsten Gundlach², Eythor Runar Eiriksson¹, Knut Conradsen¹, Anders Bjorholm Dahl¹, Jakob Andreas Bærentzen¹, Jeppe Revall Frisvad^1,*, and Henrik Aanæs¹

¹Technical University of Denmark, Department of Applied Mathematics and Computer Science, Richard Petersens Plads, 2800 Kongens Lyngby, Denmark
²Technical University of Denmark, Department of Physics, Fysikvej, 2800 Kongens Lyngby, Denmark
^†Joint primary authors
^*Corresponding author

Transparent objects require acquisition modalities that are very different from the ones used for objects with more diffuse reflectance properties. Digitizing a scene where objects must be acquired with different modalities, requires scene reassembly after reconstruction of the object surfaces. This reassembly of a scene that was picked apart for scanning seems unexplored. We contribute with a multimodal digitization pipeline for scenes that require this step of reassembly. Our pipeline includes measurement of bidirectional reflectance distribution functions and high dynamic range imaging of the lighting environment. This enables pixelwise comparison of photographs of the real scene with renderings of the digital version of the scene. Such quantitative evaluation is useful for verifying acquired material appearance and reconstructed surface geometry, which is an important aspect of digital content creation. It is also useful for identifying and improving issues in the different steps of the pipeline. In this work, we use it to improve reconstruction, apply analysis by synthesis to estimate optical properties, and to develop our method for scene reassembly.

Supplementary Visualizations

Reference

Stets, J. D., Dal Corso, A., Nielsen, J. B., Lyngby, R. A., Jensen, S. H. N., Wilm, J., Doest, M. B., Gundlach, C., Eiriksson, E. R., Conradsen, K., Dahl, A. B., Bærentzen, J. A., Frisvad, J. R., and Aanæs, H. Scene reassembly after multimodal digitization and pipeline evaluation using photorealistic rendering. Applied Optics 56(27):7679-7690, September 2017. [abstract] [video] [data] [lowres pdf] [news release] [front cover image] [editor's pick]