Multi-modal Gesture Recognition Using Skeletal Joints and Motion Trail Model

Bin Liang, Lihong Zheng

Research output: Book chapter/Published conference paperConference paper

4 Citations (Scopus)
7 Downloads (Pure)


his paper proposes a novel approach to multi-modal gesture recognition by using skeletal joints and motion trail model. The approach includes two modules, i.e. spotting and recognition. In the spotting module, a continuous gesture sequence is segmented into individual gesture intervals based on hand joint positions within a sliding window. In the recognition module, three models are combined to classify each gesture interval into one gesture category. For skeletal model, Hidden Markov Models (HMM) and Support Vector Machines (SVM) are adopted for classifying skeleton features. For depth maps and user masks, we employ 2D Motion Trail Model (2DMTM) for gesture representation to capture motion region information. SVM is then used to classify Pyramid Histograms of Oriented Gradient (PHOG) features from 2DMTM. These three models are complementary to each other. Finally, a fusion scheme incorporates the probability weights of each classifier for gesture recognition. The proposed approach is evaluated on the 2014 ChaLearn Multi-modal Gesture Recognition Challenge dataset. Experimental results demonstrate that the proposed approach using combined models outperforms single-modal approaches, and the recognition module can perform effectively on user-independent gesture recognition.
Original languageEnglish
Title of host publicationEuropean Conference on Computer Vision 2014
Place of PublicationSwitzerland
Number of pages16
Publication statusPublished - 2014
EventEuropean Conference on Computer Vision - Zurich, Switzerland
Duration: 01 Jan 201112 Sep 2014


ConferenceEuropean Conference on Computer Vision
Abbreviated titleECCV
Internet address

Fingerprint Dive into the research topics of 'Multi-modal Gesture Recognition Using Skeletal Joints and Motion Trail Model'. Together they form a unique fingerprint.

  • Cite this

    Liang, B., & Zheng, L. (2014). Multi-modal Gesture Recognition Using Skeletal Joints and Motion Trail Model. In European Conference on Computer Vision 2014 (pp. 623-638). Springer.