Emotionally Expressive Motion Controller for Virtual Character Locomotion Animations

Project for the acquisition of a Masters Degree in Computer Science and Engineering

Diogo Silva

Abstract

Style and emotional expressiveness are essential aspects of virtual character computer animation. For a virtual character to display different emotions, motion capture data conveying each desired style has to be recorded, even if the baseline motion is the same. Animators then have to refine and conjoin each recording in order to create the final animations making it a timely and costly process. Although there have been efforts made into the automatic generation of motions through Deep Reinforcement Learning techniques, the problem persists that, for each new desired emotion, reference data displaying said emotion has to be readily available and a new motion has to be learned from scratch. By combining Machine Learning with Emotion Analysis - in particular Laban Movement Analysis and the Pleasure, Arousal, Dominance Emotion State Model - we have developed a system that is capable of not only identifying the perceived emotion of virtual character locomotion animations but that also allows us to alter the character's expressed emotion in real time and without the need of additional data.

Video

...
Gallery

System and GUI
...
Synthesized Sad Policy Controlled Character (left) and Reference Motion (right)
...
Synthesized "Afraid" Motion on a Kinematic Character
...
Synthesized "Confident" Motion on a Policy-Controlled Character
...
Synthesized "Tired" Motion on a Policy-Controlled Character
...
Synthesized "Happy" Motion on a Policy-Controlled Character
...
System Architecture
...
Emotion Prediction Subsystem
...
Motion Synthesis Subsystem
...
Emotion Prediction Results
...
Motion Synthesis Heuristic Rules
...
User Emotion Classification Results for Mocap and AutoEncoder Synthesized Clips
...
User Emotion Agreence Results for Mocap and AutoEncoder Synthesized Clips
User Test Results

"Emotional Agreeance" User Tests
"Emotional Classification" User Tests
Clustered "Emotional Classification" User Tests