[Next]
[Up] [Previous]
Next:Introduction
Virtual Kathakali: Gesture-Driven Metamorphosis
Soumyadeep Paul, Sudipta N. Sinha, and Amitabha Mukerjee
Center for Robotics
Indian Institute of Technology, Kanpur
e-mail: {spaul,snsinha,amit}@iitk.ac.in
Abstract:
Training in motor skills such as athletics, dance, or gymnastics is
not possible today except in the direct presence of the coach/instructor.
This paper describes a computer vision based gesture recognition system
which is used to metamorphose the user into a Virtual person, e.g. as a
Kathakali dancer, which is graphically recreated at a near or distant location.
Thus this can be seen by an off-site coach using low-bandwidth joint-motion
data which permits real-time animation. The metamorphosis involves altering
the appearance and identity of the user and also creating a specific environment
possibly in interaction with other virtual creatures. Unlike previous approaches
to gesture based virtual reality, here i) a single monochrome camera is
used to track the arms, and ii) colour differences are used to disambiguate
situations where the arm is occluding the user's body.
A robust vision module is used to identify the user, based on very
simple binary image processing in real time which also manages to resolve
self occlusion, correct for clothing/colour and other variations among
users. Gestures are identified by locating key points at the shoulder,
elbow and wrist joint, which are then recreated in an articulated humanoid
model, which in this instance, represents a Kathakali dancer in elaborate
traditional dress. Unlike glove based or other gesture and movement tracking
systems, this application requires the user to wear no hardware devices
and is aimed at making gesture tracking simpler, cheaper, and more user
friendly.