Publication Type

Journal Article

Version

publishedVersion

Publication Date

8-2008

Abstract

Gesture plays an important role for recognizing lecture activities in video content analysis. In this paper, we propose a real-time gesture detection algorithm by integrating cues from visual, speech and electronic slides. In contrast to the conventional "complete gesture" recognition, we emphasize detection by the prediction from "incomplete gesture". Specifically, intentional gestures are predicted by the modified hidden Markov model (HMM) which can recognize incomplete gestures before the whole gesture paths are observed. The multimodal correspondence between speech and gesture is exploited to increase the accuracy and responsiveness of gesture detection. In lecture presentation, this algorithm enables the on-the-fly editing of lecture slides by simulating appropriate camera motion to highlight the intention and flow of lecturing. We develop a real-time application, namely simulated smartboard, and demonstrate the feasibility of our prediction algorithm using hand gesture and laser pen with simple setup without involving expensive hardware.

Keywords

gesture detection, lecture video, real-time simulated smartboard

Discipline

Computer Sciences | Graphics and Human Computer Interfaces

Research Areas

Intelligent Systems and Optimization

Publication

IEEE Transactions on Multimedia

Volume

10

Issue

5

First Page

926

Last Page

935

ISSN

1520-9210

Identifier

10.1109/TMM.2008.922871

Publisher

Institute of Electrical and Electronics Engineers

Download

Share

COinS