Speech-Driven Cartoon Animation with Emotions
Author(s): Yan Li, Feng Yu, Ying-Qing Xu, Eric Chang, Heung-Yeung Shum.
Proceedings: ACM Multimedia, pp. 365--371, Ottawa, Canada, Sep-Oct,
2001.
[BibTeX] 
Abstract:
In this paper, we present a cartoon face animation system
for multimedia HCI applications. We animate face cartoons
not only from input speech, but also based on emotions derived
from speech signal. Using a corpus of over 700 utterances
from different speakers, we have trained SVMs (support
vector machines) to recognize four categories of emotions:
neutral, happiness, anger and sadness. Given each
input speech phrase, we identify its emotion content as a
mixture of all four emotions, rather than classifying it into a
single emotion. Then, facial expressions are generated from
the recovered emotion for each phrase, by morphing different
cartoon templates that correspond to various emotions. To
ensure smooth transitions in the animation, we apply lowpass
filtering to the recovered (and possibly jumpy) emotion
sequence. Moreover, lip-syncing is applied to produce the
lip movement from speech, by recovering a statistical audiovisual
mapping. Experimental results demonstrate that cartoon
animation sequences generated by our system are of
good and convincing quality.