|
|
Capturing Multiple Illumination Conditions using Time and Color Multiplexing
Bert De Decker, Jan Kautz, Tom Mertens and Philippe Bekaert
CVPR 2009
(pdf)
(mov)
Many vision and graphics problems such as relighting, structured light scanning and photometric stereo, need images of a scene under a number of different illumination conditions. It is typically assumed that the scene is static. To extend such methods to dynamic scenes, dense optical flow can be used to register adjacent frames. This registration becomes inaccurate if the frame rate is too low with respect to the degree of movement in the scenes.
We present a general method that extends time multiplexing with color multiplexing in order to better handle dynamic scenes. Our method allows for packing more illumination information into a single frame, thereby reducing the number of required frames over which optical flow must be computed. Moreover, color-multiplexed frames lend themselves better to reliably computing optical flow. We show that our method produces better results compared to time-multiplexing alone. We demonstrate its application to relighting, structured light scanning and photometric stereo in dynamic scenes. |
|
|
|
Floating Textures
M. Eisemann, B. De Decker, M. Magnor, P. Bekaert, E. de Aguiar, N. Ahmed, C. Theobalt, A. Sellent
Computer Graphics Forum (Proc. Eurographics EG'08), vol. 27, no. 2, pages 409-418, April 2008
(link)
|
We present a novel multi-view, projective texture mapping technique. While previous multi-view texturing approaches lead to blurring and ghosting artefacts if 3D geometry and/or camera calibration are imprecise, we propose a texturing algorithm that warps (``floats'') projected textures during run-time to preserve crisp, detailed texture appearance. Our GPU implementation achieves interactive to real-time frame rates. The method is very generally applicable and can be used in combination with many image-based rendering methods or projective texturing applications. By using Floating Textures in conjunction with, e.g., visual hull rendering, light field rendering, or free-viewpoint video, improved rendering results are obtained from fewer input images, less accurately calibrated cameras, and coarser 3D geometry proxies. In a nutshell, the notion of Floating Textures is to correct for local texture misalignments by determining the optical flow between projected textures and warping the textures accordingly in the rendered image domain. Both steps, optical flow estimation and multi-texture warping, can be efficiently implemented on graphics hardware to achieve interactive to real-time performance. |
|
|
|
Prakash: Lighting Aware Motion Capture using Photosensing Markers and Multiplexed Illuminators
Ramesh Raskar, Hideaki Nii, Bert De Decker, Yuki Hashimoto, Jay Summet, Dylan Moore, Yong Zhao, Jonathan Westhues, Paul Dietz, John Barnwell, Shree Nayar, Masahiko Inami, Philippe Bekaert, Michael Noland, Vlad Branzoi, Erich Burns
SIGGRAPH 2007
(link)
In this paper, we present a high speed optical motion capture method which can measure three dimensional motion, orientation, and incident illumination at tagged points in a scene. We use tracking tags that can be imperceptibly embedded in attire or other objects and can work in natural lighting conditions. Our system can support an unlimited number of tags in a scene, and each tag has a unique id thus eliminating marker reacquisition issues. Our tags also provide incident illumination data which can be used when inserting synthetic elements in order to match the lighting of the scene at the time of capturing. This makes the technique ideal for on-set motion capture or the real-time broadcasting of virtual sets.
Unlike previous methods that employ high speed cameras or scanning lasers, we capture the scene appearance using the simplest possible optical devices . a light-emitting diode (LED) with a passive binary mask used as the transmitter and a photosensor used as the receiver. We strategically place a set of optical transmitters to spatio-temporally encode the volume of interest. Photosensors attached to scene points demultiplex the coded optical signals from multiple transmitters, allowing us to compute not only their location and orientation but also their incident illumination and the reflectance of the surfaces to which they are attached. We use our untethered tag system to demonstrate methods of adding special effects to captured videos that cannot be accomplished using pure vision techniques that rely on camera images. |
|
|
|
Illumination sensitive dynamic virtual sets
Bert De Decker, Hideaki Nii, Yuki Hashimoto, Dylan Moore, Jay Summet, Yong Zhao, Jonathan Westhues, Paul Dietz, John Barnwell, Masahiko Inami, Philippe Bekaert, Ramesh Raskar
Sketch at SIGGRAPH 2007
(link)
(pdf)
|
We describe a novel motion capture method that is sensitive to incident illumination. We will demonstrate the corresponding system at Emerging Technologies at Siggraph 2007. In this sketch, we focus on the key technical and implementation contributions involved in building a real-time special effects system where a prop is turned into a virtual sword with appropriate orientation and incident illumination. |
|
|
|
Prakash: Lighting-Aware Motion Capture for Dynamic Virtual Sets
Ramesh Raskar, Hideaki Nii, Bert De Decker, Yuki Hashimoto, Dylan Moore, Jay Summet, Yong Zhao, Jonathan Westhues, Paul Dietz, John Barnwell, Masahiko Inami, Philippe Bekaert
SIGGRAPH 2007 Emerging Technologies
(link)
|
This project demonstrates new methods of flexible scene capture (including motion, orientation, and incident illumination) that create a dynamic 'virtual recording set'. The system uses tracking tags that are imperceptible under attire, and inserted computer graphics elements can match the lighting on the presenter, making the technique ideal for real-time broadcast. |
|
|
|
Interactive Collision Detection for Free-Viewpoint Video
Bert De Decker, Tom Mertens, Philippe Bekaert
GRAPP 2007: Proceedings of the Second International Conference on Computer Graphics Theory and Applications
Barcelona, isbn 978-972-8865-72-6, pages 114-120, Spain, 2007
(pdf)
(avi)
(ppt without movies)
(ppt with movies)
|
We present a novel way of interacting with a virtual 3D scene in the context of free-viewpoint video. Using a multi-camera setup, our technique detects collisions between virtual objects and real objects, including people. We perform collision computations directly on the image data, as opposed to reconstructing the full geometry of the subject. This reduces implementation complexity, and moreover, yields interactive performance. We demonstrate the effectiveness of our technique by incorporating it in a rigid body simulation. The subject can interact with virtual objects and observe his or her actions while being able to adjust the viewpoint, all in real-time. |
|
|
|
Interactive Acquisition and Rendering of Humans
Bert De Decker, Philippe Bekaert
Proc. of Workshop on Content Generation and Coding for 3D-Television
Eindhoven, Netherlands, 2006-06-02
(pdf)
|
We present a distributed model-based system for interactive acquisition and rendering of free viewpoint video of humans. All current model-based systems are offline. Online systems which can be used for free viewpoint video of humans use few low resolution cameras or use only silhouette information while the method presented here can cope with high resolution images and uses both color and silhouette information. A person is captured with multiple synchronized and calibrated digital video cameras attached to a cluster of pcs. At interactive rates, the shape of the person is estimated and the person can be rendered from novel viewpoints. First the pose of the person is obtained using a real-time motion capturing algorithm. A generic mesh of a human is transformed into this pose to obtain a first approximation of the shape of the recorded person. This mesh is refined to match the input images as closely as possible by translating its vertices along their normals. The refined mesh is rendered using a distributed version of the unstructured lumigraph algorithm. |
|