Non-Photorealistic Computer Graphics Library

Empathic Painting: Interactive stylization through observed emotional state

Author(s): Maria Shugrina, Margrit Betke, John P. Collomosse.
Proceedings: NPAR '06: Proceedings of the 4th international symposium on Non-photorealistic animation and rendering, pp. 87--96, New York, NY, USA, June, ACM Press, 2006.
[BibTeX] [DOI]

Abstract:
We present the "empathic painting" -- an interactive painterly rendering whose appearance adapts in real time to reflect the perceived emotional state of the viewer. The empathic painting is an experiment into the feasibility of using high level control parameters (namely, emotional state) to replace the plethora of low-level constraints users must typically set to affect the output of artistic rendering algorithms. We describe a suite of Computer Vision algorithms capable of recognising users' facial expressions through the detection of facial action units derived from the FACS scheme. Action units are mapped to vectors within a continuous 2D space representing emotional state, from which we in turn derive a continuous mapping to the style parameters of a simple but fast segmentation-based painterly rendering algorithm. The result is a digital canvas capable of smoothly varying its painterly style at approximately 4 frames per second, providing a novel user interactive experience using only commodity hardware.