BenAV: A Bengali audio-visual corpus for visual speech recognition

Ashish Pondit, Muhammad Eshaque Ali Rukon, Anik Das, Ashad Kabir

Research output: Book chapter/Published conference paperConference paperpeer-review


Visual speech recognition (VSR) is a challenging task. It has many applications such as facilitating speech recognition when the acoustic data is noisy or missing, assisting hearing impaired people, etc. Modern VSR systems require a large amount of data to achieve a good performance. Popular VSR datasets are mostly available for the English language and none in Bengali. In this paper, we present a large-scale Bengali audio-visual dataset, named "BenAV". To the best of our knowledge, BenAV is the first publicly available large-scale dataset in the Bengali language. BenAV contains a lexicon of 50 words from 128 speakers with a total number of 26,300 utterances. We have also applied three existing deep learning based VSR models to provide a baseline performance of our BenAV dataset. We run extensive experiments in two different configurations of the dataset to study the robustness of those models and achieved 98.70% and 82.5% accuracy, respectively. We believe that this research provides a basis to develop Bengali lip reading systems and opens the doors to conduct further research on this topic.
Original languageEnglish
Title of host publicationNeural Information Processing
Subtitle of host publication28th International Conference, ICONIP 2021, Proceedings, Part II
EditorsTeddy Mantoro, Minho Lee, Media Anugerah Ayu, Kok Wai Wong, Achmad Nizar Hidayanto
Place of PublicationCham, Switzerland
Number of pages10
ISBN (Electronic)9783030922702
ISBN (Print)9783030922696
Publication statusE-pub ahead of print - 07 Dec 2021
EventThe 28th International Conference on Neural Information Processing (ICONIP 2021): ICONIP 2021 - Online, Bali, Indonesia
Duration: 08 Dec 202112 Dec 2021 (Conference website)

Publication series

NameLecture Notes in Computer Science
PublisherSpringer, Cham
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceThe 28th International Conference on Neural Information Processing (ICONIP 2021)
OtherThe 28th International Conference on Neural Information Processing (ICONIP2021) aims to provide a leading international forum for researchers, scientists, and industry professionals who are working in neuroscience, neural networks, deep learning, and related fields to share their new ideas, progresses and achievement, through its regular sessions, special sessions, tutorials, and workshops.

ICONIP 2021 will be held in online mode during December 8-12, 2021.
Internet address


Dive into the research topics of 'BenAV: A Bengali audio-visual corpus for visual speech recognition'. Together they form a unique fingerprint.

Cite this