Non-Photorealistic Computer Graphics Library

Stereoscopic Non-Photorealistic Rendering

Author(s): Efstathios Stavrakis.
PhD Thesis: Vienna University of Technology, Austria, December, 2008.
[BibTeX]

Abstract:
Communication of spatiality in 2D visual arts has been a central topic around which artistic experimentation has flourished for centuries. The inherent limitation of ``flatness'' governing most traditional visual media has proven to be fertile ground for the emergence of advanced pictorial techniques (e.g.~linear perspective) that attempt to counter it. Despite the multitude of pictorial techniques developed, handcrafted works of art (e.g.~paintings, drawings, etc.) that adequately provide monocular depth cues often fail to capture the vibrancy and plasticity of a truly three-dimensional world and, in addition, may poorly engage and immerse the observer. In contrast to these techniques, stereoscopy has been instrumental in vindicating that monocular depth cues alone were insufficient for our visual system to infer robustly depth and spatial relations from a single picture. However, while stereoscopy was adopted soon after its discovery by the scientific community, only a relatively small number of fine artists have studied and used this peculiar medium for artistic purposes. The use of stereoscopy in fine arts enabled artists to create paintings and drawings that could detach from the flat surfaces they were laid on and float directly in front of the observer's eyes, opening a whole new world of possibilities for artistic experimentation. As stunning an experience of seeing stereoscopic artworks as it is, the creation of these dual works is a rather tedious and laborious process. The artist not only has to create the artwork twice from slightly dissimilar vantage points, he is also required to preserve feature and color correspondences between the two projections with great care, without introducing artifacts that might hinder stereoscopic fusion. In the digital era, abundant computational methods have been developed to create monoscopic images that resemble artwork, using as an input either 3D models or images. Despite the high availability of such algorithms, hardly any research has been done so far in the area of artistic stereoscopic rendering from real images. The work presented in this dissertation provides a stepping stone in the direction of combining computer graphics and vision techniques to form novel image-based stereoscopic Non-Photorealistic Rendering algorithms. These algorithms can be used to transform photographic stereoscopic images into pairs of pictures that resemble stereoscopic drawings, cartoons or paintings. Central to all our algorithms is the use of stereo correspondences, calculated by using stereo matching algorithms. These correspondences, usually encoded in a disparity map, are used to propagate style that is synthesized in the geometry of one stereo view to the other, thus preserving the consistency of the texture across the two views of a stereo pair. In addition, the disparity map is utilized in order to identify image areas, known as occluded regions, that are not visible from both views simultaneously, and thus style generated in one view cannot be propagated in these areas using stereo correspondences. Instead, in these occluded regions texture synthesis procedures specifically generate stylized texture that seamlessly blends with the texture of surrounding non-occluded regions, within the same stereoscopic view. Furthermore, the artistic-looking stereoscopic image pairs generated using our algorithms provide a basis over which a set of interactive methods and tools are built to enhance the experience of viewers. We provide a way to manipulate stereoscopic space by simple adjustment of the separation of the two image projections, and we demonstrate the use of the disparity map as a means of automatically choosing optimal separation values in order to reduce eye-strain. In addition, we describe the design of a stereoscopic cursor and a magnifying glass that can be used to inspect the stereoscopic results generated by our system. Finally, we expose viewers to a new method of interacting with computer-generated stereoscopic paintings that enables them to slice through the painting and observe the understructure of the work. The novel algorithms presented in this work set the foundation to harnessing stereoscopy as an artistic medium within the context of image-based computer graphics and vision, and their results may find utility in the game, media or film industries.