Username   Password       Forgot your password?  Forgot your username? 


Short Text Classification based on Feature Extension using Information in Images

Volume 15, Number 2, February 2019, pp. 667-675
DOI: 10.23940/ijpe.19.02.p31.667675

Shengjie Zhaoa,b and Qianyun Jianga

aCollege of Electronic and Information Engineering, Tongji University, Shanghai, 200800, China
bSchool of Software Engineering, Tongji University, Shanghai, 200800, China

(Submitted on November 10, 2018; Revised on December 12, 2018; Accepted on January 5, 2019)


With the quick development and extensive application of the Internet, there is a growing desire for people to share their life or opinions on social networks, which produces a mass of short texts. Short texts are characterized by short length, sparse features, and a lack of contextual information. Thus, it is difficult for conventional methods to achieve high quality classification performance. To achieve a higher classification accuracy, this paper proposes a novel short text classification method based on feature extension by incorporating the information of the images. Specifically, we first generate a sentence that descripts the images by image caption technology, and then we combine the generated sentence with the text as the input of the classifier. Meanwhile, we introduce a similarity module in terms of the correlation between the image and the short text so as to determine whether the two sentences are combined or not. Simulation results show that our proposed model significantly outperforms the state-of-the-art methods in terms of classification accuracy.


References: 287

        1. B. Sriram, D. Fuhry, E. Demir, H. Ferhatosmanoglu, and M. Demirbas, “Short Text Classification in Twitter to Improve Information Filtering,” in Proceedings of International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 841-842, 2010
        2. M. Wang, L. Lin, and F. Wang, “Improving Short Text Classification through Better Feature Space Selection,” in Proceedings of International Conference on Computational Intelligence and Security, pp. 120-124, 2014
        3. D. Bollegala, M. Ishizuka, and Y. Matsuo, “Measuring Semantic Similarity Between Words using Web Search Engines,” in Proceedings of International Conference on World Wide Web, pp. 757-766, Banff, Alberta, Canada, May 2007
        4. X. Hu, N. Sun, C. Zhang, and T. S. Chua, “Exploiting Internal and External Semantics for the Clustering of Short Texts using World Knowledge,” in Proceedings of ACM Conference on Information and Knowledge Management, pp. 919-928, 2009
        5. A. Paccanaro and G. E. Hinton, “Learning Distributed Representations of Concepts using Linear Relational Embedding,” IEEE Transactions on Knowledge & Data Engineering, Vol. 13, No. 2, pp. 232-244, 2002
        6. X. Zhang and B. Wu, “Short Text Classification based on Feature Extension using the n-Gram Model,” in Proceedings of 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), pp.710-716, IEEE, 2015
        7. Christopher Bonnett, “Classifying E-Commerce Products based on Images and Text,” (
        8. T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, “Distributed Representations of Words and Phrases and their Compositionality,” in Proceedings of International Conference on Neural Information Processing Systems, pp. 3111-3119, 2013
        9. J. Pennington, R. Socher, and C. Manning, “Glove: Global Vectors for Word Representation,” in Proceedings of Conference on Empirical Methods in Natural Language Processing, pp. 1532-1543, 2014
        10. D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent Dirichlet Allocation,” J Machine Learning Research Archive, Vol. 3, pp. 993-1022, 2003
        11. A. Bordes, J. Weston, R. Collobert, and Y. Bengio, “Learning Structured Embeddings of Knowledge Bases,” AAAI, Vol. 6. No. 1, 2011
        12. R. Johnson and T. Zhang, “Effective Use of Word Order for Text Categorization with Convolutional Neural Networks,” arXiv preprint arXiv:1412.1058, 2014
        13. Y. Kim, “Convolutional Neural Networks for Sentence Classification,” arXiv preprint arXiv:1408.5882, 2014
        14. C. N. D. Santos and M. Gattit, “Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts,” in Proceedings of International Conference on Computational Linguistics, 2014
        15. R. Kiros, R. Salakhutdinov, and R. Zemel, “Multimodal Neural Language Models,” in Proceedings of International Conference on Machine Learning, pp. 595-603, 2014
        16. K. Cho, B. Van Merriënboer, C Gulcehre, et al., “Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation,” arXiv preprint arXiv:1406.1078, 2014
        17. S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, Vol. 9, No. 8, pp. 1735-1780, 1997
        18. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan, “Show and Tell: A Neural Image Caption Generator,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156-3164, 2015
        19. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. “Show and Tell: Lessons Learned from the 2015 MSCOCO Image Captioning Challenge,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 652-663, 2016
        20. M. Surdeanu, M. Ciaramita, and H. Zaragoza, “Learning to Rank Answers to Non-Factoid Questions from Web Collections,” Computational Linguistics, Vol. 37, No. 2, pp. 351-383, 2011
        21. N. Kalchbrenner, E. Grefenstette, and P. Blunsom, “A Convolutional Neural Network for Modelling Sentences,” arXiv preprint arXiv:1404.2188, 2014
        22. A. Severyn and A. Moschitti. “Modeling Relational Information in Question-Answer Pairs with Convolutional Neural Networks,” arXiv preprint arXiv:1604.01178, 2016
        23. O. Abdel-Hamid, A. R. Mohamed, H. Jiang, and G. Penn, “Applying Convolutional Neural Networks Concepts to Hybrid NN-HMM Model for Speech Recognition,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 4277-4280, 2012
        24. C. Szegedy, L. Wei, J. Yangqing, et al., “Going Deeper with Convolutions,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-9, 2015
        25. W. Zaremba, I. Sutskever, and O. Vinyals, “Recurrent Neural Network Regularization,” Eprint Arxiv, 2014
        26. L. M. Surhone, M. T. Tennoe, and S. F. Henssonow, “Long Short Term Memory,” Betascript Publishing, 2010
        27. A. Severyn and A. Moschitti, “Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks,” in Proceedings of the International ACM SIGIR Conference, pp. 373-382, 2015
        28. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, et al., “Microsoft coco: Common Objects in Context,” in Proceedings of European Conference on Computer Vision, Springer, pp. 740-755, 2014




        Please note : You will need Adobe Acrobat viewer to view the full articles.Get Free Adobe Reader

        This site uses encryption for transmitting your passwords.