Username   Password       Forgot your password?  Forgot your username? 

An Attention-Based Syntax-Tree and Tree-LSTM Model for Sentence Summarization

Volume 13, Number 5, September 2017 - Paper 20  - pp. 775-782
DOI: 10.23940/ijpe.17.05.p20.775782

Wenfeng Liua,b, Peiyu Liua,c,*, Yuzhen Yangb, Yaling Gaod, Jing Yia,e

aSchool of Information Science and Engineering, Shandong Normal University, Jinan, 250014, China
b
Department of Computer and Information Engineering, Heze University, Heze, 274015, China
c
Shandong Provincial Key Laboratory for Distributed Computer Software Novel Technology, Jinan,250014, China
d
Ruipu Peony Industrial Technology Development Co., Ltd, 274015, China
e
School of Computer Science and Technology, Shandong Jianzhu University, Jinan, 250101, China

(Submitted on January 29, 2017; Revised on April 12, 2017; Accepted on June 23, 20177)

Abstract:

Generative Summarization is of great importance in understanding large-scale textual data. In this work, we propose an attention-based Tree-LSTM model for sentence summarization, which utilizes an attention-based syntactic structure as auxiliary information. Thereinto, block-alignment is used to align the input and output syntax blocks, while inter-alignment is used for alignment of words within that of block pairs. To some extent, block-alignment can prevent structural deviations on the long sentences and inter-alignment is capable of increasing the flexibility of the generation in the blocks. This model can be easily trained to end-to-end mode and deal with any length of the input sentences. Compared with several relatively strong baselines, our model has achieved the state-of-art on DUC-2004 shared task.

 

References: 21

    1. D. Andor, C. Alberti, D. Weiss, et al., "Globally normalized transition-based neural networks", arXiv preprint arXiv: 1603.06042, 2016.
    2. Ayana, S. Shen, Z. Liu, et al., "Neural headline generation with minimum risk training", arXiv preprint arXiv: 1604.01904, 2016.
    3. D. Bahdanau, K. Cho, Y. Bengio, "Neural machine translation by jointly learning to align and translate", arXiv preprint arXiv: 1409.0473, 2014.
    4. D. Chen, C.D. Manning, "A fast and accurate dependency parser using neural networks", in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 740-750, 2014.
    5. J. Clarke, M. Lapata, "Global inference for sentence compression: An integer linear programming approach", Journal of Artificial Intelligence Research, vol.31, pp. 399-429, 2008.
    6. T. Cohn, M. Lapata, "Sentence compression beyond word deletion", in Proceedings of the International Conference on Computational Linguistics, pp. 137-144, 2008.
    7. K. Filippova, Y. Altun, "Overcoming the lack of parallel data in sentence compression", in Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1481-1491, 2013.
    8. K. Knight, D. Marcu, "Summarization beyond sentence extraction: A probabilistic approach to sentence compression", Artificial Intelligence, vol.139, no.1, pp. 91-107, 2002.
    9. P. Koehn, H. Hoang, A. Birch et al., "Moses: Open source toolkit for statistical machine translation", in Proceedings of the 45th annual meeting of the ACL on interactive poster and demonstration sessions, pp. 177-180, 2007.
    10. K. Lopyrev, "Generating news headlines with recurrent neural networks", arXiv preprint arXiv: 1512.01712, 2015.
    11. C. D. Manning, M. Surdeanu, J. Bauer, et al., "The stanford corenlp natural language processing toolkit", in Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55-60, 2014.
    12. C. D. Manning, P. Raghavan, "Introduction to information retrieval", People's Posts and Telecommunications Press, pp. 824-825, 2010
    13. J. Nivre, "Algorithms for Deterministic Incremental Dependency Parsing", Computational Linguistics, vol.34, no.4, pp. 513-553, 2008.
    14. A. M. Rush, S. Chopra, J. Weston, "A neural attention model for abstractive sentence summarization", arXiv: 1509. 00685v2, 2015.
    15. K. S. Tai, R. Socher, C. D. Manning, "Improved Semantic Representations from Tree-Structured Long Short-Term Memory Networks", arXiv preprint arXiv: 1503.00075, 2015.
    16. D. Weiss, C. Alberti, M. Collins, "Structured training for neural network transition-based parsing." arXiv preprint arXiv: 1506.06158, 2015.
    17. K. Woodsend, Y. Feng, M. Lapata, "Generation with quasi-synchronous grammar", in Proceedings of the Association for Computational Linguistics, pp. 513-523, 2010.
    18. S. Wubben, A. V. D Bosch, E. Krahmer, "Sentence simplification by monolingual machine translation", in Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, vol.1, pp. 1015-1024, 2012.
    19. D. Zajic, B. Dorr, R. Schwartz, "Bbn/umd at duc-2004: Topiary", in Proceedings of the HLT-NAACL 2004 Document Understanding Workshop, Boston, pp. 112-119, 2004.
    20. Y. Zhang, J. Nivre, "Transition-based dependency parsing with rich non-local features", in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pp. 740-750, 2011.
    21. H. Zhou, Y. Zhang, S. Huang, et al., "A Neural Probabilistic Structured-Prediction Model for Transition-Based Dependency Parsing", Association for Computational Linguistics, vol. 1, pp. 1213-1222, 2015

     

       

      Please note : You will need Adobe Acrobat viewer to view the full articles.Get Free Adobe Reader

       
      This site uses encryption for transmitting your passwords. ratmilwebsolutions.com