An Innovative Approach of Bangla Text Summarization by Introducing Pronoun Replacement and Improved Sentence Ranking
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
This paper proposes an automatic method to summarize Bangla news document. In the proposed approach,pronoun replacement is accomplished for the first time to minimize the dangling pronoun from summary. After replacing pronoun, sentences are ranked using term frequency, sentence frequency, numerical figuresand title words. If two sentences have at least 60% cosine similarity, the frequency of the larger sentence isincreased, and the smaller sentence is removed to eliminate redundancy. Moreover, the first sentence isincluded in summary always if it contains any title word. In Bangla text, numerical figures can be presentedboth in words and digits with a variety of forms. All these forms are identified to assess the importance ofsentences. We have used the rule-based system in this approach with hidden Markov model and Markovchain model. To explore the rules, we have analyzed 3,000 Bangla news documents and studied some Banglagrammar books. A series of experiments are performed on 200 Bangla news documents and 600 summaries (3summaries are for each document). The evaluation results demonstrate the effectiveness of the proposedtechnique over the four latest methods.
Abstract
This paper proposes an automatic method to summarize Bangla news document. In the proposed approach,pronoun replacement is accomplished for the first time to minimize the dangling pronoun from summary. After replacing pronoun, sentences are ranked using term frequency, sentence frequency, numerical figuresand title words. If two sentences have at least 60% cosine similarity, the frequency of the larger sentence isincreased, and the smaller sentence is removed to eliminate redundancy. Moreover, the first sentence isincluded in summary always if it contains any title word. In Bangla text, numerical figures can be presentedboth in words and digits with a variety of forms. All these forms are identified to assess the importance ofsentences. We have used the rule-based system in this approach with hidden Markov model and Markovchain model. To explore the rules, we have analyzed 3,000 Bangla news documents and studied some Banglagrammar books. A series of experiments are performed on 200 Bangla news documents and 600 summaries (3summaries are for each document). The evaluation results demonstrate the effectiveness of the proposedtechnique over the four latest methods.
참고문헌 (51)
- D. Ai, "Automatic text summarization based on latent semantic indexing", Journal of Artificial Life and Robotics, Springer, 15(1), pp.25~29 (2010)
- R. Ferreira, "A multi-document summarization system based on statistics and linguistic treatment", Journal of Expert Systems with Applications, 41(13), pp.5780~5787 (2014)
- M. M. Haque, "Literature review of automatic multiple documents text summarization", International Journal of IAS, 3(1), pp.121~129 (2013)
- M. M. Haque, "Literature review of automatic single document text summarization using NLP", International Journal of IAS, 3(3), pp.857~865 (2013)
- K. Sarkar, "Bengali text summarization by sentence extraction", Proceedings of International Conference on Business and Information Management (ICBIM-2012), pp.233~245 (2012)
- K. Sarkar, "An approach to summarizing Bengali news documents", Proceedings of the International Conference on Advances in Computing, Communications and Informatics, pp.857~862 (2012)
- H. P. Luhn, "The automatic creation of literature abstracts", IBM Journal of Research and Development, 2(2), pp.159~165 (1958)
- H. P. Edmundson, "New methods in automatic extracting", Journal of the ACM, 16(2), pp.264~285 (1969)
- M. I. Efat, "Automated Bangla text summarization by sentence scoring and ranking", Proceedings of International Conference on Informatics, Electronics & Vision (ICIEV), pp.1~5 (2013)
- "Banglapedia: the National Encyclopedia of Bangladesh" (2003), Asiatic Society of Bangladesh
- G. Miller, "WordNet : a lexical database for English", Communications of the Association for Computing Machinery(CACM), 38(11), pp.39~41 (1995)
- "Indradhanush WordNet Development for the Bengali Language" (2017)[URL]
- M. A. Karim, "Technical Challenges and Design Issues in Bangla Language Processing" (2013), Information Science Reference
- "Bangla language and research on Bangla language processing: its motivation and impact!" (2008)[URL]
- M. M. Haque, "Automatic Bengali news documents summarization by introducing sentence frequency and clustering", Proceedings of 18th International Conference on Computer and Information Technology (ICCIT), pp.156~160 (2015)
- V. Gupta, "A survey of text summarization extractive techniques", Journal of Emerging Technologies in Web Intelligence, 2(3), pp.258~268 (2010)
- H. Saggion, "Multi-source, Multilingual Information Extraction and Summarization", pp.3~21 (2013), Springer
- E. Canhasi, "Weighted archetypal analysis of the multi-element graph for query-focused multi-document summarization", Expert Systems with Applications, 41(2), pp.535~543 (2014)
- A. M. Azmia, "A text summarizer for Arabic", Journal of Computer Speech & Language, 26(4), pp.260~273 (2012)
- T. Islam, "Bhasa : a corpus-based information retrieval and summariser for Bengali text", Proceedings of the 7th International Conference on Computer and Information Technology (2004)
- N. Uddin, "A study on text summarization techniques and implement few of them for Bangla language", Proceedings of 10th International conference on Computer and Information Technology, pp.1~4 (2007)
- A. Das, "Topic-based Bengali opinion summarization", Proceedings of the 23rd International Conference on Computational Linguistics (COILING10), pp.232~240 (2010)
- K. Sarkar, "A keyphrase-based approach to text summarization for English and Bengali documents", International Journal of Technology Diffusion(IJTD), 5(2), pp.28~38 (2014)
- S. R. El-Beltagy, "KP-Miner : a keyphrase extraction system for English and Arabic documents", Journal Information Systems, 34(1), pp.132~144 (2009)
- "ROUGE 2.0: a Java package for automatic summary evaluation"[URL]
- "List of stop words for Bengali language" (2016)[URL]
- M. Islam, "A light weight stemmer for Bengali and its use in spelling checker", Center for Research on Bangla Language Processing, ~ (2007)
- "Bengali POS Tagger"[URL]
- A. Das, "Knowledge Sharing Event-4: Task 2: Building Electronic Dictionary" (2010), Knowledge Sharing Event
- M. Chowdhury, "Bangla Vasar Byakaran" (2000), Ideal Publishers
- H. Mamud, "Bangla Vasar Byakaran O Rachanariti" (2011), The Atlas Publishing House
- "Name of occupation in largest job site"[URL]
- "Gpedia"[URL]
- A. Ekbal, "Named entity recognition in Bengali: a conditional random field approach", Proceedings of International Joint Conference on Natural Language Processing, pp.589~594 (2008)
- Z. R. Siddiqui, "English-Bangla Dictionary" (2011), Bangla Academy
- G. M. Kiron, "Ajker Bishaw (General Knowledge, Bangladesh and International Affairs)" (2014), Premier Publications
- "Bengali names"[URL]
- "Post office of Bangladesh"[URL]
- L. Gravano, "Using qgrams in a DBMS for approximate string processing", IEEE Data Engineering Bulletin, 24(4), pp.28~34 (2001)
- G. Salton, "Term-weighting approaches in automatic text retrieval", Information Processing &Management, 25(5), pp.513~523 (1988)
- A. Abuobieda, "Text summarization features selection method using pseudo genetic-based model", Proceedings of International Conference on Information Retrieval & Knowledge Management, pp.193~197 (2012)
- M. A. Fattah, "GA, MR, FFNN, PNN and GMM based models for automatic text summarization", Computer Speech and Language, 23(1), pp.126~144 (2009)
- D. R. Radev, "Introduction to the special issue on summarization", Journal of Computational Linguistics, 28(4), pp.399~408 (2002)
- "Rule based system"[URL]
- "Bangla Natural Language Processing Community"[URL]
- R. Ferreira, "A four dimension graph model for automatic text summarization", Proceedings of the 2013 IEEE, ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), pp.2013[DOI]
- J. Chen, "Summarization of scientific documents by detecting common facts in citations", Future Generation Computer Systems, 32, pp.246~252 (2014)
- C. Lin, "Automatic evaluation of summaries using n-gram co-occurrence statistics", Proceedings of the Human Technology Conference (HLT-NAACL-2003), pp.71~78 (2003)
- S. Hariharan, "Enhanced graph based approach for multi document summarization", The International Arab Journal of Information Technology, 10(4), pp.334~341 (2013)