Introduction

One application of gesture recognition that is beginning to emerge is off-site training for motor skills, e.g. in activities such as athletics, surgery, theater/dancing, or gymnastics. Here the user's motions can be transmitted using some low-bandwidth representation (e.g. joint angles or facial expressions), and the instructor or coach at the remote site can visualize the student using a local graphics model at real-time animation rate, and provide appropriate feedback, possibly illustrating the correct procedure through a similar virtual metamorphosis channel. For example, a renowned master in the Kathakali dance form[+] may be able to provide personalized feedback to a student far away - the sensation of being co-located in the same virtual space makes communication much more natural. Furthermore, the master (or disciple) has the ability to zoom in on a particular part of the performance or view the scene from a particular vantage point, or to have the actions repeated in slower speeds.

Of course, other usual Virtual Reality applications such as full body interaction in a virtual space, as in games or advanced chat rooms, can also be conducted with such a system. Figure 1 shows the basic setup that would be needed.

Early approaches to gesture modeling used specialized arm motion detection sensors [14]. Such sensors encumber the user and impose constraints on their motion to a certain extent. The camera based model provides a simpler, more flexible, and far cheaper alternative to other approaches. However, with the camera the body pose is not directly available, and considerable effort is needed in image processing. Different parts of this problem have been tackled for many years now:

• The DigitEyes system Rehg/Kanade 94 ([12]) recovered a detailed kinematic description of the hand using a 27 degrees of freedom full hand model and one or two cameras.
• Markov model based gesture identification [1,13].
• Body tracking and behaviour interpretation [16].

Figure 2: The User and the Model. The User pose as seen by the camera (from which the arm pose is to be detected), and the final Kathakali Dancer Model as displayed to the user.

Combining gesture recognition with graphics reconstruction provides a virtual space where the user's action can be reflected. Applications in this genre include games [4,6], Virtual interaction spaces [3], remote tele-operation [5], and Virtual Metamorphosis [8]. Our application is in the metamorphosis category where the user is metamorphosed as a Kathakali dancer in a virtual environment. The following section gives a brief outline of the techniques used in the paper.