Learn2Write: Augmented reality and machine learning-based mobile app to learn writing

Md. Nahidul Islam Opu, M. D. Rakibul Islam, Ashad Kabir, Md. Sabir Hossain, Mainul Islam

Research output: Contribution to journalArticlepeer-review

11 Citations (Scopus)
135 Downloads (Pure)

Abstract

Augmented reality (AR) has been widely used in education, particularly for child education. This paper presents the design and implementation of a novel mobile app, Learn2Write, using machine learning techniques and augmented reality to teach alphabet writing. The app has two main features: (i) guided learning to teach users how to write the alphabet and (ii) on-screen and AR-based handwriting testing using machine learning. A learner needs to write on the mobile screen in on-screen testing, whereas AR-based testing allows one to evaluate writing on paper or a board in a real world environment. We implement a novel approach to use machine learning for AR-based testing to detect an alphabet written on a board or paper. It detects the handwritten alphabet using
our developed machine learning model. After that, a 3D model of that alphabet appears on the screen with its pronunciation/sound. The key benefit of our approach is that it allows the learner to use a handwritten alphabet. As we have used marker-less augmented reality, it does not require a static image as a marker. The app was built with ARCore SDK for Unity. We further evaluated and
quantified the performance of our app on multiple devices.
Original languageEnglish
Article number4
Pages (from-to)1-12
Number of pages12
JournalComputers
Volume11
Issue number1
DOIs
Publication statusPublished - 27 Dec 2021

Fingerprint

Dive into the research topics of 'Learn2Write: Augmented reality and machine learning-based mobile app to learn writing'. Together they form a unique fingerprint.

Cite this