TY - JOUR
T1 - Multimodal big data affective analytics
T2 - A comprehensive survey using text, audio, visual and physiological signals
AU - Shoumy, Nusrat J.
AU - Ang, Li Minn
AU - Seng, Kah Phooi
AU - Rahaman, D. M.Motiur
AU - Zia, Tanveer
PY - 2020/1/1
Y1 - 2020/1/1
N2 - Affective computing is an emerging multidisciplinary research field that is increasingly drawing the attention of researchers and practitioners in various fields, including artificial intelligence, natural language processing, cognitive and social sciences. Research in affective computing includes areas such as sentiment, emotion, and opinion modelling. The internet is an excellent source of data required for sentiment analysis, such as customer reviews of products, social media, forums, blogs, etc. Most of these data, called big data, are unstructured and unorganized. Hence there is a strong demand for developing suitable data processing techniques to process these rich and valuable data to produce useful information. Early surveys on sentiment and emotion recognition in the literature have been limited to discussions using text, audio, and visual modalities. So far, to the author's knowledge, a comprehensive survey combining physiological modalities with these other modalities for affective computing has yet to be reported. The objective of this paper is to fill the gap in this surveyed area. The usage of physiological modalities for affective computing brings several benefits in that the signals can be used in different environmental conditions, more robust systems can be constructed in combination with other modalities, and it has increased anti-spoofing characteristics. The paper includes extensive reviews on different frameworks and categories for state-of-the-art techniques, critical analysis of their performances, and discussions of their applications, trends and future directions to serve as guidelines for readers towards this emerging research area.
AB - Affective computing is an emerging multidisciplinary research field that is increasingly drawing the attention of researchers and practitioners in various fields, including artificial intelligence, natural language processing, cognitive and social sciences. Research in affective computing includes areas such as sentiment, emotion, and opinion modelling. The internet is an excellent source of data required for sentiment analysis, such as customer reviews of products, social media, forums, blogs, etc. Most of these data, called big data, are unstructured and unorganized. Hence there is a strong demand for developing suitable data processing techniques to process these rich and valuable data to produce useful information. Early surveys on sentiment and emotion recognition in the literature have been limited to discussions using text, audio, and visual modalities. So far, to the author's knowledge, a comprehensive survey combining physiological modalities with these other modalities for affective computing has yet to be reported. The objective of this paper is to fill the gap in this surveyed area. The usage of physiological modalities for affective computing brings several benefits in that the signals can be used in different environmental conditions, more robust systems can be constructed in combination with other modalities, and it has increased anti-spoofing characteristics. The paper includes extensive reviews on different frameworks and categories for state-of-the-art techniques, critical analysis of their performances, and discussions of their applications, trends and future directions to serve as guidelines for readers towards this emerging research area.
KW - Affective applications
KW - Affective computing
KW - Multimodal fusion
KW - Sentiment analysis
KW - Sentiment databases
UR - http://www.scopus.com/inward/record.url?scp=85074459721&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85074459721&partnerID=8YFLogxK
U2 - 10.1016/j.jnca.2019.102447
DO - 10.1016/j.jnca.2019.102447
M3 - Review article
AN - SCOPUS:85074459721
SN - 1084-8045
VL - 149
SP - 1
EP - 26
JO - Journal of Network and Computer Applications
JF - Journal of Network and Computer Applications
M1 - 102447
ER -