Bangla Sign Language recognition using convolutional neural network

Farhad Yasir, P. W.C. Prasad, Abeer Alsadoon, A. Elchouemi, Sasikumaran Sreedharan

Research output: Book chapter/Published conference paperConference paper

4 Citations (Scopus)

Abstract

This paper presents a learning based approach to Bangla Sign Language(BdSL) recognition using the convolutional neural network. In our proposed method, a virtual reality-based hand tracking controller known as Leap motion controller (LMC) has introduced to track the continuous motion of the hands. LMC provides a skeletal model of the hand with appropriate data of hand position, orientation, rotation, fingertips, grabbing and more non-linear features. This controller preprocessed all the motion features and provides error free data. This machine calibrates with the environment and builds a virtual hand in a space. LMC also calculates the rotation, orientation, and textures from hands to determine and to extract hand gesture. In the next process, an efficient method is established to proceed a sequence of frames for positional hand gestures and summarize them to a shorter and more generalized sequence of lines and curves which are added to a Hidden Markov Model. For each sign of expression, we considered a start and an end point of state and segmented the state transitions into segmented HMM. In the segmentation, we assumed the state scope of the hidden variables is discrete. The transition probabilities controlled the way of hidden state at a distinct time. If there is a histogram difference in any state, the transition state moved to new frame to achieve a new sign expression. If there is no hand gesture in the frame, the state has ended by moving to the end point of the model. In the end point, we evaluated the desired hand gesture for recognition. After evaluation, hand gesture data set are proceeded over the convolutional neural network (CNN) and built a decision network. Each neuron is built up by calculating the dot product of extracted features in the dataset. In CNN, a single vector of hand gesture data is received and connected through a series of hidden layers and in the end point computed as a single vector loss function. Each feature is considered as a hidden layer. Determining the least loss function, the network recognizes the expected sign expression. In our experiment, we considered training data first to create the neurons in our network as a supervised way. We achieved significant results from our basic sign expressions in a 3% rate of error where without distortion the rate reduced to 2%. This is an enormous achievement in the Bangla sign language recognition method.
Original languageEnglish
Title of host publicationProceedings of the 2017 International Conference on Intelligent Computing, Instrumentation and Control Technologies
Subtitle of host publicationICICICT 2017
Place of PublicationUSA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages49-53
Number of pages5
ISBN (Electronic)9781509061068
DOIs
Publication statusPublished - 23 Apr 2018
Event2017 International Conference on Intelligent Computing, Instrumentation and Control Technologies: ICICICT 2017 - Vimal Jyothi Engineering College, Kannur, India
Duration: 06 Jul 201707 Jul 2017
http://vjaei.com/icicict2017/index.html (Conference website)

Conference

Conference2017 International Conference on Intelligent Computing, Instrumentation and Control Technologies
Abbreviated titleIntelligent Systems for Smart World
CountryIndia
CityKannur
Period06/07/1707/07/17
Otherhe Department of Electronics and Instrumentation Engineering of Vimal Jyothi Engineering College, one of the prominent Engineering Institution in Kerala, takes immense pleasure in organizing an International Conference On Intelligent Computing, Instrumentation And Control Technologies. The theme of the conference is Intelligent Systems for Smart World. The Conference will be a platform to focus on the core technological developments on Instrumentation and other emerging technologies to be held on the 6th and 7th July 2017 at Vimal Jyothi Engineering College. ICICICT-2017 aims to provide an opportune forum and vibrant platform for Researchers, Academicians, Scientists and Industrial Practitioners to share their original research work, patents, findings and practical development experiences on the specific new challenges and emerging issues confronting the nation.
Internet address

Fingerprint Dive into the research topics of 'Bangla Sign Language recognition using convolutional neural network'. Together they form a unique fingerprint.

  • Cite this

    Yasir, F., Prasad, P. W. C., Alsadoon, A., Elchouemi, A., & Sreedharan, S. (2018). Bangla Sign Language recognition using convolutional neural network. In Proceedings of the 2017 International Conference on Intelligent Computing, Instrumentation and Control Technologies: ICICICT 2017 (pp. 49-53). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICICICT1.2017.8342533