TY - JOUR
T1 - A novel enhanced convolution neural network with extreme learning machine
T2 - Facial emotional recognition in psychology practices
AU - Banskota, Nitesh
AU - Alsadoon, Abeer
AU - Prasad, P. W.C.
AU - Dawoud, Ahmed
AU - Rashid, Tarik A.
AU - Alsadoon, Omar Hisham
N1 - Publisher Copyright:
© 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.
PY - 2023/2
Y1 - 2023/2
N2 - Facial emotional recognition is one of the essential tools used by recognition psychology to diagnose patients. Face and facial emotional recognition are areas where machine learning is excelling. Facial Emotion Recognition in an unconstrained environment is an open challenge for digital image processing due to different environments, such as lighting conditions, pose variation, yaw motion, and occlusions. Deep learning approaches have shown significant improvements in image recognition. However, accuracy and time still need improvements. This research aims to improve facial emotion recognition accuracy during the training session and reduce processing time using a modified Convolution Neural Network Enhanced with Extreme Learning Machine (CNNEELM). The proposed system consists of an optical flow estimation technique that detects the motion of change in facial expression and extracts peak images from video frames for image pre-processing. The system entails (CNNEELM) improving the accuracy in image registration during the training session. Furthermore, the system recognizes six facial emotions – happy, sad, disgust, fear, surprise, and neutral with the proposed CNNEELM model. The study shows that the overall facial emotion recognition accuracy is improved by 2% than the state of art solutions with a modified Stochastic Gradient Descent (SGD) technique. With the Extreme Learning Machine (ELM) classifier, the processing time is brought down to 65 ms from 113 ms, which can smoothly classify each frame from a video clip at 20fps. With the pre-trained InceptionV3 model, the proposed CNNEELM model is trained with JAFFE, CK+, and FER2013 expression datasets. The simulation results show significant improvements in accuracy and processing time, making the model suitable for the video analysis process. Besides, the study solves the issue of the large processing time required to process the facial images.
AB - Facial emotional recognition is one of the essential tools used by recognition psychology to diagnose patients. Face and facial emotional recognition are areas where machine learning is excelling. Facial Emotion Recognition in an unconstrained environment is an open challenge for digital image processing due to different environments, such as lighting conditions, pose variation, yaw motion, and occlusions. Deep learning approaches have shown significant improvements in image recognition. However, accuracy and time still need improvements. This research aims to improve facial emotion recognition accuracy during the training session and reduce processing time using a modified Convolution Neural Network Enhanced with Extreme Learning Machine (CNNEELM). The proposed system consists of an optical flow estimation technique that detects the motion of change in facial expression and extracts peak images from video frames for image pre-processing. The system entails (CNNEELM) improving the accuracy in image registration during the training session. Furthermore, the system recognizes six facial emotions – happy, sad, disgust, fear, surprise, and neutral with the proposed CNNEELM model. The study shows that the overall facial emotion recognition accuracy is improved by 2% than the state of art solutions with a modified Stochastic Gradient Descent (SGD) technique. With the Extreme Learning Machine (ELM) classifier, the processing time is brought down to 65 ms from 113 ms, which can smoothly classify each frame from a video clip at 20fps. With the pre-trained InceptionV3 model, the proposed CNNEELM model is trained with JAFFE, CK+, and FER2013 expression datasets. The simulation results show significant improvements in accuracy and processing time, making the model suitable for the video analysis process. Besides, the study solves the issue of the large processing time required to process the facial images.
KW - Convolution neural network
KW - Cross-entropy loss
KW - Extreme learning machine
KW - Log-likelihood estimator
KW - Optical flow estimation
KW - Stochastic gradient descent
UR - http://www.scopus.com/inward/record.url?scp=85135465125&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85135465125&partnerID=8YFLogxK
U2 - 10.1007/s11042-022-13567-8
DO - 10.1007/s11042-022-13567-8
M3 - Article
AN - SCOPUS:85135465125
SN - 1380-7501
VL - 82
SP - 6479
EP - 6503
JO - Multimedia Tools and Applications
JF - Multimedia Tools and Applications
IS - 5
ER -