Sequential deep learning for action recognition with synthetic multi-view data from depth maps

Bin Liang, Lihong Zheng, Xinying Li

Research output: Book chapter/Published conference paperConference paper

1 Citation (Scopus)

Abstract

Recurrent neural network (RNN) has proven successful recently in action recognition. However, depth sequences are of high dimensionality and contain rich human dynamics, which makes traditional RNNs difficult to capture complex action information. This paper addresses the problem of human action recognition from sequences of depth maps using sequential deep learning. The proposed method first synthesizes multi-view depth sequences by rotating 3D point clouds from depth maps. Each depth sequence is then split into short-term temporal segments. For each segment, a multi-view depth motion template (MVDMT), which compresses the segment to a motion template, is constructed for short-term multi-view action representation. The MVDMT effectively characterizes the multi-view appearance and motion patterns within a short-term duration. Convolutional Neural Network (CNN) models are leveraged to extract features from MVDMT, and a CNN-RNN network is subsequently employed to learn an effective representation for sequential patterns of the multi-view depth sequence. The proposed multi-view sequential deep learning framework can simultaneously capture spatial-temporal appearance and motion features in the depth sequence. The proposed method has been evaluated on the MSR Action3D and MSR Action Pairs datasets, achieving promising results compared with the state-of-the-art methods based on depth data.
Original languageEnglish
Title of host publicationData Mining - 16th Australasian Conference, AusDM 2018, Revised Selected Papers
EditorsYanchang Zhao, Graco Warwick, David Stirling, Chang-Tsun Li, Yun Sing Koh, Rafiqul Islam, Zahidul Islam
PublisherSpringer-Verlag London Ltd.
Chapter28
Pages360-371
Number of pages12
ISBN (Electronic)9789811366611
ISBN (Print)9789811366604
DOIs
Publication statusPublished - 2019
Event16th Australasian Conference on Data Mining, AusDM 2018 - Charles Sturt University , Bathurst, Australia
Duration: 28 Nov 201830 Nov 2018
https://ausdm18.ausdm.org/

Publication series

NameCommunications in Computer and Information Science
Volume996
ISSN (Print)1865-0929

Conference

Conference16th Australasian Conference on Data Mining, AusDM 2018
CountryAustralia
CityBathurst
Period28/11/1830/11/18
Internet address

Fingerprint Dive into the research topics of 'Sequential deep learning for action recognition with synthetic multi-view data from depth maps'. Together they form a unique fingerprint.

  • Cite this

    Liang, B., Zheng, L., & Li, X. (2019). Sequential deep learning for action recognition with synthetic multi-view data from depth maps. In Y. Zhao, G. Warwick, D. Stirling, C-T. Li, Y. S. Koh, R. Islam, & Z. Islam (Eds.), Data Mining - 16th Australasian Conference, AusDM 2018, Revised Selected Papers (pp. 360-371). (Communications in Computer and Information Science; Vol. 996). Springer-Verlag London Ltd.. https://doi.org/10.1007/978-981-13-6661-1_28