Explainable Multimodal Deep Learning Model for Cyberbullying Detection (EMDL-CBD)

Mena Mohammed  Abood; Maha A.  Al-Bayati

doi:10.36371/port.2024.3.6

Authors

Mena Mohammed Abood Department of Computer Science, College of Science, Al-Mustansiriyah University, Baghdad, Iraq.
Maha A. Al-Bayati Department of Computer Science, College of Science, Al-Mustansiriyah University, Baghdad, Iraq.

Abstract References Downloads Cite Statistics Share Information

The increased use of social media and the internet is leading to an increase in cyberbully vulnerabilities as well as daily usage. Cyberbullying is a deliberate, aggressive behavior that can be committed by an individual or organization. It occurs when people communicate, post, and distribute damaging, false, or unfavorable content online. For individuals impacted, it results in emotional and mental health issues. Therefore, it is imperative to create automated techniques for the detection and prevention of cyberbullying. The majority of the research done on cyberbullying detection in recent years has been on text-based analysis. The two most significant media in incidents of cyberbullying are text and visual. This paper presents An Explainable Multimodal Deep Learning Model for Cyberbullying Detection include three steps The first step involves collecting datasets from different resources, which include images and their captions with binary classes (bullying and non-bullying). The second step applies two techniques of XAI: CNN+GradCam to analyze input images and produce visual explanations, and LSTM+LRP to analyze and interpret input text. The third step employs two techniques of data fusion (early and late). Final step represents the evaluation performance of the EMDL-CBD model based on a set of accuracy metrics

Keywords:

Cyberbullying detection, multimodal data, Grad-Cam, LRP, LSTM

[1] J. L. Wu and C. Y. Tang, "Classifying the severity of cyberbullying incidents by using a hierarchical squashing-attention network," Applied Sciences, vol. 12, no. 7, p. 3502, 2022.

[2] N. Haydar and B. N. Dhannoon, "A comparative study of cyberbullying detection in social media for the last five years," Al-Nahrain Journal of Science, vol. 26, no. 2, pp. 47-55, 2023.

[3] M. H. Obaid, S. K. Guirguis, and S. M. Elkaffas, "Cyberbullying detection and severity determination model," IEEE Access, 2023.

[4] N. A. Azeez, S. Misra, O. I. Lawal, and J. Oluranti, "Identification and detection of cyberbullying on Facebook using machine learning algorithms," Journal of Cases on Information Technology (JCIT), vol. 23, no. 4, pp. 1-21, 2021.

[5] M. A. Khan, S. Kadry, P. Parwekar, R. Damaševičius, A. Mehmood, J. A. Khan, and S. R. Naqvi, "Human gait analysis for osteoarthritis prediction: a framework of deep learning and kernel extreme learning machine," Complex Intell. Syst., 2021. Available: https://doi.org/10.1007/s40747-020-00244-2.

[6] R. Jadhav, G. Chaudhari, and S. Rane, "Cyber bullying detection," Int. J. Creative Res. Thoughts (IJCRT), pp. 1–6, 2023.

[7] N. Vishwamitra, H. Hu, F. Luo, and L. Cheng, "Towards understanding and detecting cyberbullying in real-world images," in 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA), 2021.

[8] B. A. Talpur and D. O’Sullivan, "Cyberbullying severity detection: A machine learning approach," PLoS ONE, vol. 15, no. 10, Art. no. e0240924, 2020.

[9] A. Muneer and S. M. Fati, "A comparative analysis of machine learning techniques for cyberbullying detection on Twitter," Future Internet, vol. 12, no. 11, p. 187, 2020.

[10] H. Hitkul, R. R. Shah, P. Kumaraguru, and S. Satoh, "Maybe look closer? Detecting trolling prone images on Instagram," in 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), pp. 448–456, IEEE, 2019.

[11] H. Rosa, D. Matos, R. Ribeiro, L. Coheur, and J. P. Carvalho, "A 'deeper' look at detecting cyberbullying in social networks," in 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–8, IEEE, 2018.

[12] K. Hara, T. Kojima, K. Kutsukake, H. Kudo, and N. Usami, "3D CNN and grad-CAM based visualization for predicting generation of dislocation clusters in multicrystalline silicon," APL Machine Learning, vol. 1, no. 3, 2023.

[13] F. Vaquerizo-Villar, G. C. Gutiérrez-Tobal, E. Calvo, D. Álvarez, L. Kheirandish-Gozal, F. Del Campo, et al., "An explainable deep-learning model to stage sleep states in children and propose novel EEG-related patterns in sleep apnea," Computers in Biology and Medicine, vol. 165, p. 107419, 2023.

[14] L. D. Quach, K. N. Quoc, A. N. Quynh, N. Thai-Nghe, and T. G. Nguyen, "Explainable deep learning models with gradient-weighted class activation mapping for smart agriculture," IEEE Access, vol. 11, pp. 83752-83762, August 2023.

[15] Kareem, M. R., "Image analysis and detection of olive leaf diseases using recurrent neural networks," Al-Mustansiriyah Journal of Science, vol. 35, no. 1, pp. 60-65, 2024.

[16] M. H. Al-Tai, B. M. Nema, and A. Al-Sherbaz, "Deep learning for fake news detection: Literature review," Al-Mustansiriyah Journal of Science, vol. 34, no. 2, pp. 70-81, 2023.

[17] M. T. Hasan, M. A. E. Hossain, M. S. H. Mukta, A. Akter, M. Ahmed, and S. Islam, "A review on deep-learning-based cyberbullying detection," Future Internet, vol. 15, no. 5, p. 179, 2023.

[18] A. Dewani, M. A. Memon, and S. Bhatti, "Cyberbullying detection: advanced preprocessing techniques & deep learning architecture for Roman Urdu data," J. Big Data, vol. 8, no. 1, p. 160, 2021.

[19] A. Bennetot et al., "A practical tutorial on explainable ai techniques," arXiv, 2021, arXiv:2111.14260.

[20] T. R. Dieter and H. Zisgen, "Evaluation of the Explanatory Power Of Layer-wise Relevance Propagation using Adversarial Examples," Neural Processing Letters, vol. 55, no. 7, pp. 8531-8550, 2023.

[21] R. Singh, "Understanding Image Classification Tasks Through Layerwise Relevance Propogation," in 2022 IEEE 18th International Conference on Intelligent Computer Communication and Processing (ICCP), September 2022, pp. 199-203.

[22] N. M. Khassaf and S. H. Shaker, "Image Retrieval based Convolutional Neural Network," Al-Mustansiriyah Journal of Science, vol. 31, no. 4, pp. 43-54, 2020

[23] Afrah U Mosaa, Waleed A Mahmoud Al-Jawher “A proposed Hyper-Heuristic optimizer Nesting Grey Wolf Optimizer and COOT Algorithm for Multilevel Task” Journal Port Science Research, Vol. 6, PP. 310,317, 2023.

[24] [24] Walid A Mahmoud, Majed E Alneby, Wael H Zayer “2D-multiwavelet transform 2D-two activation function wavelet network-based face recognition” J. Appl. Sci. Res, Vol. 6, Issue 6, PP. 1019-1028, 2010.

[25] Maryam I Al-Khuzaie, Waleed A Mahmoud Al-Jawher “Enhancing Medical Image Classification: A Deep Learning Perspective with Multi Wavelet Transform” Journal Port Science Research, Vol. 6, Issue 4, PP. 365-373, 2023.

[26] A. H. Salman, W. A. Mahmoud Al-Jawher “A Hybrid Multiwavelet Transform with Grey Wolf Optimization Used for an Efficient Classification of Documents” International Journal of Innovative Computing 13 (1-2), 55-60, 2022.

[27] Saadi M Saadi and Waleed A Mahmoud Al-Jawher “Image Fake News Prediction Based on Random Forest and Gradient-boosting Methods” Journal Port Science Research, Vol. 6, Issue 4, PP. 357-364, 2023.

[28] Ahmed Hussein Salman, Waleed Ameen Mahmoud Al-Jawher “A Hybrid Multiwavelet Transform with Grey Wolf Optimization Used for an Efficient Classification of Documents” International Journal of Innovative Computing, Vol. 13, Issue 1-2, PP. 55-60, 2022.

[29] Sarah H Awad Waleed A Mahmoud Al-Jawher “Precise Classification of Brain Magnetic Resonance Imaging (MRIs) using Gray Wolf Optimization (GWO)” HSOA Journal of Brain & Neuroscience Research, Volume 6, Issue 1, Pages 100021, 2022.

[30] W. A. Mahmoud, Jane Jaleel Stephan and A. A. W. Razzak “Facial Expression Recognition from Video Sequence Using Self Organizing Feature Map” Journal port Science Researchو TRANSACTION ON ENGINEERING, TECHNOLOGY AND THEIR APPLICATIONS, Vol. 4, Issue 2, 2021.

[31] Dihin, R. Al-Jawher, Waleed and Al-Shemmary “Implementation of The Swin Transformer and Its Application In Image Classification” Journal Port Science Research, vol. 6, Issue 4, PP. 318-331. 2023

[32] W. A. Mahmoud & Z. Ragib “Face Recognition Using PCA and Optical Flow” Engineering Journal, Vol. 13, Issue 1, PP. 35-47, 2007.

[33] Waleed A Mahmoud Al-Jawher, Sarah H Awad “A proposed brain tumor detection algorithm using Multi wavelet Transform (MWT)” Materials Today: Proceedings, Volume 65, Pages 2731-2737, 2022.

[34] Rasha Ali Dihin, Waleed A Mahmoud Al-Jawher, Ebtesam N AlShemmary “Diabetic Retinopathy Image Classification Using Shift Window Transformer”, International Journal of Innovative Computing, Vol. 13, Issue 1-2, PP. 23-29, 2022.

[35] AHM Al-Heladi, W. A. Mahmoud, HA Hali, AF Fadhel “Multispectral Image Fusion using Walidlet Transform” Advances in Modelling and Analysis B, Volume 52, Iss. 1-2, pp. 1-20, 2009.

[36] Maryam I Mousa Al-Khuzaay, Waleed A Mahmoud Al-Jawher, “New Proposed Mixed Transforms: CAW and FAW and Their Application in Medical Image Classification” International Journal of Innovative Computing, Volume 13, Issue 1-2, Pages 15-21, 2022.

PDF

Abood , M. M. ., & Al-Bayati , M. A. . (2024). Explainable Multimodal Deep Learning Model for Cyberbullying Detection (EMDL-CBD). Journal Port Science Research, 7(3). https://doi.org/10.36371/port.2024.3.6