Profanity and hate speech detection

Teh, Phoey Lee * and Cheng, Chi-Bin (2020) Profanity and hate speech detection. International Journal of Information and Management Sciences, 31 (3). pp. 227-246. ISSN 1017-1819

[img] Text
Teh Phoey Lee Preprint - Profanity and Hate Speech Detection.pdfx - Accepted Version
Available under License Creative Commons Attribution Non-commercial.

Download (334kB)
Official URL:


Profanity, often found in today’s online social media, has been used to detect online hate speech. The aims of this study were to investigate the profanity usage on Twitter by different groups of users, and to quantify the effectiveness of using profanity in detecting hate speech. Tweets from three English-speaking countries, Australia, Malaysia, and the United States, were collected for data analysis. Statistical hypothesis tests were performed to justify the difference of profanity usage among the three countries, and a probability estimation procedure was formulated based on Bayes theorem to quantify the effectiveness of profanity-based methods in hate speech detection. Three deep learning methods, long short-term memory (LSTM), bidirectional LSTM (BLSTM), and bidirectional encoder representations from transformers (BERT) are further used to evaluate the effect of profanity screening on building classification model. Our experimental results show that the effectiveness of using profanity in detecting hate speech is questionable. Nevertheless, the results also show that for Australia tweets, where profanity is more associated with hatred, profanity-based methods in hate speech detection could be effective and profanity screening can address the class imbalance issue in hate speech detection. This is evidenced by the performances of using deep learning methods on the profanity screened data of Australia data, which achieved a classification f1-score greater than 0.84.

Item Type: Article
Uncontrolled Keywords: Profanity; Hate speech; Tweets; Bayes theorem; Deep learning
Subjects: H Social Sciences > HM Sociology
Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: Others > Non Sunway Academics
Sunway University > School of Engineering and Technology [formerly School of Science and Technology until 2020] > Dept. Computing and Information Systems
Depositing User: Dr Janaki Sinnasamy
Related URLs:
Date Deposited: 04 Dec 2020 03:55
Last Modified: 30 Jul 2021 08:18

Actions (login required)

View Item View Item