RNN LSTM-based Deep Hybrid Learning Model for Text Classification using Machine Learning Variant XGBoost

doi:10.23940/ijpe.22.08.p2.545551

Int J Performability Eng ›› 2022, Vol. 18 ›› Issue (8): 545-551.doi: 10.23940/ijpe.22.08.p2.545551

Previous Articles Next Articles

RNN LSTM-based Deep Hybrid Learning Model for Text Classification using Machine Learning Variant XGBoost

Sandhya Alagarsamy^a,* and Visumathi James^b

^aDepartment of Computer Science Engineering, Sathyabama Institute of Science and Technology, Chennai, 600100, India;
^bDepartment of Computer Science Engineering, Veltech Rangarajan Dr Sagunthala R&D Institute of Science and Technology, Chennai, 600085, India

Submitted on ; Revised on ; Accepted on
Contact: * E-mail address: sandhyalagar@gmail.com
About author:Sandhya Alagarsamy has completed the Bachelor’s degree in Computer Science and Engineering from SASTRA Deemed University, Master degree in Information Technology from Sathyabama University, Chennai, India. She is persuing her doctorate in the field of Big Data Analytics in Sathyabama University. She has 5 years of Industry experience and 7 years of teaching experience. She has published more than 5 papers in conferences and journals. Her current areas of interest include Bigdata, Artificial Intelligence and Deep learning.
Dr. Visumathi James has completed the Bachelor’s degree in Computer Science and Engineering from Manonmaniam Sundaranar University, Master and Ph.D. degree in Computer Science and Engineering from Sathyabama University, Chennai, India. She is working as Professor in the department of Computer Science and Engineering, Vel Tech Rangarajan Dr.Sagunthala R&D Institute of Science and Technology, Chennai, India. She has 20 years of teaching experience. She has published more than 75 papers in conferences and journals. Her current areas of interest include Network Security, Data mining, Bigdata, Cloud Computing and Artificial Intelligence.

Abstract

Abstract: Text classification is an emerging area in Natural Language Processing (NLP). On the other hand, traditional text classification methods need to be improved due to the complexity and semantic nature in text. In this paper, we build a hybrid deep learning model using Deep Learning (DL) and Machine learning (ML) models. This work combines two traditional neural networks namely Long Short-Term Memory (LSTM) and Recurrent Neural Network (RNN) to extract the features from the text document. LSTM preserves the historical information for text sequences and extracts the features using the RNN structure. The extracted features are used to run on machine learning classification algorithms like AdaBoost and XGBoost to perform the final prediction. Thereby the proposed Deep Hybrid Model eliminates the fully connected classification layers from a typical Deep Learning model. The performance of proposed model is measured with other models and the results show that the deep hybrid model provides about 12% increased results in terms of accuracy in text classification.

Key words: deep learning, machine learning, text classification, hybrid model, RNN, LSTM

Sandhya Alagarsamy and Visumathi James. RNN LSTM-based Deep Hybrid Learning Model for Text Classification using Machine Learning Variant XGBoost [J]. Int J Performability Eng, 2022, 18(8): 545-551.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

1. Fan Y., Lu X., Li D., andLiu Y.Video-based emotion recognition using CNN-RNN and C3D hybrid networks,ACM International Conference on Multimodal Interaction. ACM, pp. 445-450, 2016.
2. Fan, Y. Chen, Y., Lin, K., and Huang, C. Using deep neural networks to evaluate the system reliability of manufacturing networks, International Journal of Performability Engineering, vol. 17, no. 7, pp. 600-608, 2021.
3. Mikolov T., Karafiat M., Burget L., andCernocky J.Recurrent neural network based language model, The 11thInternational Speech Communication Association, Japan, pp. 235-241, 2010.
4. Rao, A. and Spasojevic, N.Actionable and political text classification using word embeddings and LSTM, Computer Science, ArXiv, abs/1607.02501, 2016.
5. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E. Hierarchical attention networks for document classification, In Proceedings of the 2016 Conference of the North American Chapter Of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA, 12-17 June, pp. 1480-1489, 2016.
6. Goldberg Y.Natural language processing based on deep learning, Beijing, Mechanical Industry Press, 2018.
7. Zhen, W. and Mao-ting, G. Design and implementation of image recognition algorithm based on convolutional neural networks, Modern Computer, vol. 20, pp. 61-66, 2015.
8. Zoph, B. and Le, Q.Neural architecture search with reinforcement learning,ArXiv, doi:10.48550/ARXIV.1611.01578, 2017.
9. Zhang S., Li X., Zong M. and Zhu X.Learning k, for KNN Classification, Acm Transactions on Intelligent Systems & Technology, vol. 8, no. 3, pp. 43, 2017.
10. Kim, J. Ji,H. Oh, S. Hwang, S. Park, E., and Pobil, A. A deep hybrid learning model for customer repurchase behavior, Journal of Retailing and Consumer Services, vol. 59, 2021, 102381,ISSN 0969-6989, https://doi.org/10.1016/j.jretconser.2020.102381, 2021.
11. Younas K. Z., Niu Z., Nyamawe A., Haq I.A deep hybrid model for recommendation by jointly leveraging ratings, reviews and metadata information", Engineering Applications of Artificial Intelligence, vol. 97, ISSN 0952-1976, https://doi.org/10.1016/j.engappai.2020.104066, 2021.
12. Li J., Wu C., Song R., Xie W., andGao X.Deep hybrid 2-D-3-D CNN based on dual second-order attention with camera spectral sensitivity prior for spectral super-resolution,In IEEE Transactions on Neural Networks and Learning Systems, doi: 10.1109/TNNLS.2021.3098767, 2021.
13. Ma, Y. Peng, H., Khan, T., Cambria, E., and Hussain, A. Sentic LSTM: A hybrid network for targeted aspect-based sentiment analysis, Cognitive Computation, vol. 10, pp. 639-650, 2018.
14. Qin P., Xu W., andGuo J.An empirical convolutional neural network approach for semantic relation classification, Neurocomputing, vol. 190, pp. 1-9, 2016.
15. Zhang, H. Gan, W. and Jiang, B. Machine learning and lexicon based methods for sentiment classification: a survey,In Proceedings of the 11th Web Information System and Application Conference, Tianjin, pp. 262-265, 2014.
16. Giatsoglou M., Vozalis M., Diamantaras K., Vakali A. Sarigiannidis,G. and Chatzisavvas, K.Sentiment analysis leveraging emotions and word embeddings, Expert Systems with Applications, vol. 69, pp. 214-224, 2017.
17. Ren, Y. Zhang, Y. Zhang, M. and Ji, D. Improving twitter sentiment classification using topic-enriched multi-prototype word embeddings,In Proceedings of 30th AAAI Conference on Artificial Intelligence, Phoenix, pp. 3038-3044, 2016.
18. Ravi, K. and Ravi, V.A survey on opinion mining and sentiment analysis: tasks, approaches and applications, KnowledgeBased Systems, vol. 89, pp. 14-46, 2015.
19. Araque O.,Corcuera-Platas, I.Sánchez-Rada, J. and Iglesias, C. Enhancing deep learning sentiment analysis with ensemble techniques in social applications, Expert Systems with Applications, vol. 77, pp. 236-246, 2017.
20. Severyn, A. and Moschitti, A.Twitter sentiment analysis with deep convolutional neural networks, In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, New York, pp. 959-962, 2015.
21. Mikolov, T. Grave, E. Bojanowski, P. Puhrsch, C. and Joulin, A. Advances in Pre-training distributed word representations,In Proceedings of the 11th International Conference on Language Resources and Evaluation, Miyazaki, pp. 52-55, 2018.
22. Lauren, P., Qu, G. Zhang, F. and Lendasse, A.Discriminant document embeddings with an extreme learning machine for classifying clinical narratives, Neurocomputing, vol. 277, pp. 129-138, 2018.
23. Zhou, F. Jin, L. and Dong, J. A review of convolutional neural networks, Chinese Journal of Computers, vol. 40, no. 6, pp. 1229-1251, 2017.

[1]	Janarthanan Sekar and Ganesh Kumar T. Hyperparameter Tuning in Deep Learning-Based Image Classification to Improve Accuracy using Adam Optimization [J]. Int J Performability Eng, 2023, 19(9): 579-586.
[2]	Sanjay Razdan, Himanshu Gupta, and Ashish Seth. D-SVM: A Deep Support Vector Machine Model with Different Kernel Types for Improved Intrusion Detection Performance [J]. Int J Performability Eng, 2023, 19(9): 598-606.
[3]	Aashita Rajput, Muskan Yadav, Sachin Yadav, Megha Chhabra, and Arun Prakash Agarwal. Patch-Based Breast Cancer Histopathological Image Classification using Deep Learning [J]. Int J Performability Eng, 2023, 19(9): 607-623.
[4]	C. Rohith Bhat and Madhusundar Nelson. Artificial Intelligence Based Credit Card Fraud Detection for Online Transactions Optimized with Sparrow Search Algorithm [J]. Int J Performability Eng, 2023, 19(9): 624-632.
[5]	Kavita Pandey, and Dhiraj Pandey. Real-Time Crop Disease Detection and Remedial Suggestion through Deep Learning-based Smartphone Application [J]. Int J Performability Eng, 2023, 19(8): 491-498.
[6]	Savita Khurana, Gaurav Sharma, and Bhawna Sharma. Hybrid Machine Learning Model for Load Prediction in Cloud Environment [J]. Int J Performability Eng, 2023, 19(8): 507-515.
[7]	K. Eswara Rao, Bala Murali Pydi, T. Panduranga Vital, P. Annan Naidu, U. D. Prasann, and T. Ravikumar. An Advanced Machine Learning Approach for Student Placement Prediction and Analysis [J]. Int J Performability Eng, 2023, 19(8): 536-546.
[8]	Babaljeet Kaur and Shalli Rani. Are the Customers Receiving Exact Recommendations from the E-Commerce Companies? Towards the Identification of Gray Sheep Users Using Personality Parameters [J]. Int J Performability Eng, 2023, 19(7): 425-433.
[9]	Kshitij Kumar Sinha, Manoj Mathur, and Arun Sharma. Suitability Index Prediction for Residential Apartments Through Machine Learning [J]. Int J Performability Eng, 2023, 19(7): 434-442.
[10]	Manpreet Kaur and Shalli Rani. Recommender System: Towards Identification of Shilling Attacks in Rating System Using Machine Learning Algorithms [J]. Int J Performability Eng, 2023, 19(7): 443-451.
[11]	Srishti Bhugra and Puneet Goswami. Exploratory Review of Machine Learning-Based Software Component Reusability Prediction [J]. Int J Performability Eng, 2023, 19(7): 452-461.
[12]	Harsha Gaikwad, Sanil Gandhi, Arvind Kiwelekar, and Manjushree Laddha. Analyzing Brain Signals for Predicting Students’ Understanding of Online Learning: A Machine Learning Approach [J]. Int J Performability Eng, 2023, 19(7): 462-470.
[13]	Rakesh Kumar, Sunny Arora, Ashima Arya, Neha Kohli, Vaishali Arya, and Ekta Singh. Ensemble Learning for Appraising English Text Readability using Gompertz Function [J]. Int J Performability Eng, 2023, 19(6): 388-396.
[14]	Manvi Khatri and Ajay Sharma. Deep Learning Approach based on Iris, Face, and Palmprint Fusion for Multimodal Biometric Recognition System [J]. Int J Performability Eng, 2023, 19(6): 407-416.
[15]	Pranshu Kumar Soni and Leema Nelson. PCP: Profit-Driven Churn Prediction using Machine Learning Techniques in Banking Sector [J]. Int J Performability Eng, 2023, 19(5): 303-311.

RNN LSTM-based Deep Hybrid Learning Model for Text Classification using Machine Learning Variant XGBoost

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended 0