Feature selection approach for Twitter sentiment analysis and text classification based on Chi-Square and Naïve Bayes

S. Paudel; P. W.C. Prasad; Abeer Alsadoon; Md Rafiqul Islam; Amr Elchouemi

doi:10.1007/978-3-319-98776-7_30

Feature selection approach for Twitter sentiment analysis and text classification based on Chi-Square and Naïve Bayes

S. Paudel, P. W.C. Prasad, Abeer Alsadoon, Md Rafiqul Islam, Amr Elchouemi

Research output: Book chapter/Published conference paper › Conference paper › peer-review

4 Citations (Scopus)

Abstract

With the rapid growth of web and mobile technology, Social networking services like Twitter are widely used, resulting in large amounts of data being generated daily in social networking sites. Efficient Sentiment analysis of such data is very important for a range of applications and improvement of accuracy in detecting sentiment is the main aim of this research. This report examines the combination of a Chi-Squared feature selection algorithm, k-mean clustering and TF-IDF for attribute weighting based on Naïve Bayes, for classification of text and sentiment in communications generated on Twitter. This approach is compared with other approaches based on Naïve Bayes to give an account of their relative strengths and weaknesses. When running experiments on multi-domain twitter datasets, results indicate that the proposed method shows superior performance across a range of. The main aim of this research is to enhance the performance of the Naïve Bayes classifier using a feature selection technique.

Original language	English
Title of host publication	International Conference on Applications and Techniques in Cyber Security and Intelligence ATCI 2018
Subtitle of host publication	Applications and techniques in cyber security and intelligence
Editors	Mohammed Atiquzzaman, Zheng Xu, Jemal Abawajy, Kim-Kwang Raymond Choo, Rafiqul Islam
Publisher	Springer-Verlag London Ltd.
Pages	281-298
Number of pages	18
Volume	842
ISBN (Electronic)	978-3-319-98776-7
ISBN (Print)	9783319987750
DOIs	https://doi.org/10.1007/978-3-319-98776-7_30
Publication status	Published - 2019
Event	International Conference of Applications and Techniques in Cyber Intelligence, ATCI 2018 - Shanghai Univeristy, Shanghai, China Duration: 11 Jul 2018 → 13 Jul 2018 https://web.archive.org/web/20180909204616/http://www.atci2018.com/index.html https://researchoutput.csu.edu.au/admin/files/37226362/37225002_Published_Paper.pdf (proceedings preface and index)

Publication series

Name	Advances in Intelligent Systems and Computing
Volume	842

Conference

Conference	International Conference of Applications and Techniques in Cyber Intelligence, ATCI 2018
Country/Territory	China
City	Shanghai
Period	11/07/18 → 13/07/18
Internet address	https://web.archive.org/web/20180909204616/http://www.atci2018.com/index.html https://researchoutput.csu.edu.au/admin/files/37226362/37225002_Published_Paper.pdf (proceedings preface and index)

Access to Document

10.1007/978-3-319-98776-7_30

Cite this

Paudel, S., Prasad, P. W. C., Alsadoon, A., Islam, M. R., & Elchouemi, A. (2019). Feature selection approach for Twitter sentiment analysis and text classification based on Chi-Square and Naïve Bayes. In M. Atiquzzaman, Z. Xu, J. Abawajy, K.-K. R. Choo, & R. Islam (Eds.), International Conference on Applications and Techniques in Cyber Security and Intelligence ATCI 2018: Applications and techniques in cyber security and intelligence (Vol. 842, pp. 281-298). (Advances in Intelligent Systems and Computing; Vol. 842). Springer-Verlag London Ltd.. https://doi.org/10.1007/978-3-319-98776-7_30

Paudel, S. ; Prasad, P. W.C. ; Alsadoon, Abeer et al. / Feature selection approach for Twitter sentiment analysis and text classification based on Chi-Square and Naïve Bayes. International Conference on Applications and Techniques in Cyber Security and Intelligence ATCI 2018: Applications and techniques in cyber security and intelligence. editor / Mohammed Atiquzzaman ; Zheng Xu ; Jemal Abawajy ; Kim-Kwang Raymond Choo ; Rafiqul Islam. Vol. 842 Springer-Verlag London Ltd., 2019. pp. 281-298 (Advances in Intelligent Systems and Computing).

@inproceedings{664d5378dc9948209ee77434b5c9e491,

title = "Feature selection approach for Twitter sentiment analysis and text classification based on Chi-Square and Na{\"i}ve Bayes",

abstract = "With the rapid growth of web and mobile technology, Social networking services like Twitter are widely used, resulting in large amounts of data being generated daily in social networking sites. Efficient Sentiment analysis of such data is very important for a range of applications and improvement of accuracy in detecting sentiment is the main aim of this research. This report examines the combination of a Chi-Squared feature selection algorithm, k-mean clustering and TF-IDF for attribute weighting based on Na{\"i}ve Bayes, for classification of text and sentiment in communications generated on Twitter. This approach is compared with other approaches based on Na{\"i}ve Bayes to give an account of their relative strengths and weaknesses. When running experiments on multi-domain twitter datasets, results indicate that the proposed method shows superior performance across a range of. The main aim of this research is to enhance the performance of the Na{\"i}ve Bayes classifier using a feature selection technique.",

keywords = "Chi-squared, Feature selection, Na{\"i}ve Bayes, TF-IDF, Twitter sentiment analysis",

author = "S. Paudel and Prasad, {P. W.C.} and Abeer Alsadoon and Islam, {Md Rafiqul} and Amr Elchouemi",

note = "Includes bibliographical references.; International Conference of Applications and Techniques in Cyber Intelligence, ATCI 2018 ; Conference date: 11-07-2018 Through 13-07-2018",

year = "2019",

doi = "10.1007/978-3-319-98776-7_30",

language = "English",

isbn = "9783319987750",

volume = "842",

series = "Advances in Intelligent Systems and Computing",

publisher = "Springer-Verlag London Ltd.",

pages = "281--298",

editor = "Mohammed Atiquzzaman and Zheng Xu and Jemal Abawajy and Choo, {Kim-Kwang Raymond} and Rafiqul Islam",

booktitle = "International Conference on Applications and Techniques in Cyber Security and Intelligence ATCI 2018",

address = "Germany",

url = "https://web.archive.org/web/20180909204616/http://www.atci2018.com/index.html, https://researchoutput.csu.edu.au/admin/files/37226362/37225002_Published_Paper.pdf",

}

Paudel, S, Prasad, PWC, Alsadoon, A, Islam, MR & Elchouemi, A 2019, Feature selection approach for Twitter sentiment analysis and text classification based on Chi-Square and Naïve Bayes. in M Atiquzzaman, Z Xu, J Abawajy, K-KR Choo & R Islam (eds), International Conference on Applications and Techniques in Cyber Security and Intelligence ATCI 2018: Applications and techniques in cyber security and intelligence. vol. 842, Advances in Intelligent Systems and Computing, vol. 842, Springer-Verlag London Ltd., pp. 281-298, International Conference of Applications and Techniques in Cyber Intelligence, ATCI 2018, Shanghai, China, 11/07/18. https://doi.org/10.1007/978-3-319-98776-7_30

Feature selection approach for Twitter sentiment analysis and text classification based on Chi-Square and Naïve Bayes. / Paudel, S.; Prasad, P. W.C.; Alsadoon, Abeer et al.
International Conference on Applications and Techniques in Cyber Security and Intelligence ATCI 2018: Applications and techniques in cyber security and intelligence. ed. / Mohammed Atiquzzaman; Zheng Xu; Jemal Abawajy; Kim-Kwang Raymond Choo; Rafiqul Islam. Vol. 842 Springer-Verlag London Ltd., 2019. p. 281-298 (Advances in Intelligent Systems and Computing; Vol. 842).

Research output: Book chapter/Published conference paper › Conference paper › peer-review

TY - GEN

T1 - Feature selection approach for Twitter sentiment analysis and text classification based on Chi-Square and Naïve Bayes

AU - Paudel, S.

AU - Prasad, P. W.C.

AU - Alsadoon, Abeer

AU - Islam, Md Rafiqul

AU - Elchouemi, Amr

N1 - Includes bibliographical references.

PY - 2019

Y1 - 2019

N2 - With the rapid growth of web and mobile technology, Social networking services like Twitter are widely used, resulting in large amounts of data being generated daily in social networking sites. Efficient Sentiment analysis of such data is very important for a range of applications and improvement of accuracy in detecting sentiment is the main aim of this research. This report examines the combination of a Chi-Squared feature selection algorithm, k-mean clustering and TF-IDF for attribute weighting based on Naïve Bayes, for classification of text and sentiment in communications generated on Twitter. This approach is compared with other approaches based on Naïve Bayes to give an account of their relative strengths and weaknesses. When running experiments on multi-domain twitter datasets, results indicate that the proposed method shows superior performance across a range of. The main aim of this research is to enhance the performance of the Naïve Bayes classifier using a feature selection technique.

AB - With the rapid growth of web and mobile technology, Social networking services like Twitter are widely used, resulting in large amounts of data being generated daily in social networking sites. Efficient Sentiment analysis of such data is very important for a range of applications and improvement of accuracy in detecting sentiment is the main aim of this research. This report examines the combination of a Chi-Squared feature selection algorithm, k-mean clustering and TF-IDF for attribute weighting based on Naïve Bayes, for classification of text and sentiment in communications generated on Twitter. This approach is compared with other approaches based on Naïve Bayes to give an account of their relative strengths and weaknesses. When running experiments on multi-domain twitter datasets, results indicate that the proposed method shows superior performance across a range of. The main aim of this research is to enhance the performance of the Naïve Bayes classifier using a feature selection technique.

KW - Chi-squared

KW - Feature selection

KW - Naïve Bayes

KW - TF-IDF

KW - Twitter sentiment analysis

UR - https://link.springer.com/chapter/10.1007/978-3-319-98776-7_30#citeas

U2 - 10.1007/978-3-319-98776-7_30

DO - 10.1007/978-3-319-98776-7_30

M3 - Conference paper

AN - SCOPUS:85056824047

SN - 9783319987750

VL - 842

T3 - Advances in Intelligent Systems and Computing

SP - 281

EP - 298

BT - International Conference on Applications and Techniques in Cyber Security and Intelligence ATCI 2018

A2 - Atiquzzaman, Mohammed

A2 - Xu, Zheng

A2 - Abawajy, Jemal

A2 - Choo, Kim-Kwang Raymond

A2 - Islam, Rafiqul

PB - Springer-Verlag London Ltd.

T2 - International Conference of Applications and Techniques in Cyber Intelligence, ATCI 2018

Y2 - 11 July 2018 through 13 July 2018

ER -

Paudel S, Prasad PWC, Alsadoon A, Islam MR, Elchouemi A. Feature selection approach for Twitter sentiment analysis and text classification based on Chi-Square and Naïve Bayes. In Atiquzzaman M, Xu Z, Abawajy J, Choo KKR, Islam R, editors, International Conference on Applications and Techniques in Cyber Security and Intelligence ATCI 2018: Applications and techniques in cyber security and intelligence. Vol. 842. Springer-Verlag London Ltd. 2019. p. 281-298. (Advances in Intelligent Systems and Computing). Epub 2018 Nov 5. doi: 10.1007/978-3-319-98776-7_30

Feature selection approach for Twitter sentiment analysis and text classification based on Chi-Square and Naïve Bayes

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this