Video analytics for customer emotion and satisfaction at contact centers

Kah Phooi Seng, Li-Minn Ang

    Research output: Contribution to journalArticlepeer-review

    22 Citations (Scopus)


    Due to the high levels of competition in a global market, companies have put more emphasis on building strong customer relationships and increasing customer satisfaction levels. With technological improvements in information and communication technologies, a highly anticipated key contributor to improve the customer experience and satisfaction in service episodes is through the application of video analytics, such as to evaluate the customer’s emotions over the full service cycle. Currently, emotion recognition from video is a challenging research area. One of the most effective solutions to address this challenge is to utilize both the audio and visual components as two sources contained in the video data to make an overall assessment of the emotion. The combined use of audio and visual data sources presents additional challenges, such as determining the optimal data fusion technique prior to classification. In this paper, we propose an audio–visual emotion recognition system to detect the universal six emotions (happy, angry, sad, disgust, surprise, and fear) from video data. The detected customer emotions are then mapped and translated to give customer satisfaction scores. The proposed customer satisfaction video analytics system can operate over video conferencing or video chat. The effectiveness of our proposal is verified through numerical results.
    Original languageEnglish
    Pages (from-to)266-278
    Number of pages13
    JournalIEEE Transactions on Human-Machine Systems
    Issue number3
    Publication statusPublished - 02 May 2017


    Dive into the research topics of 'Video analytics for customer emotion and satisfaction at contact centers'. Together they form a unique fingerprint.

    Cite this