The Errors Analysis of Natural Language Generation - A Case Study of Topic-to-Essay Generation

• DOI number:10.1109/CIS52066.2020.00027
• Affiliation of Author(s):西南交通大学
• Journal:2020 16th International Conference on Computational Intelligence and Security (CIS)
• Place of Publication:Guangxi, China
• Key Words:Annotations,Error analysis,Natural languages,Neural networks,Coherence,Manuals
• Abstract:Although natural language generation (NLG) has achieved great success, there are still many problems with the generated text, if humans carefully examine it. To analyze the problems of NLG, we use manual evaluation methods to annotate and analyze the text generated by NLG. According to the analysis results, we can understand the defects of NLG in-depth, comprehensively, and accurately. Further, these provide cues for future improvement. In this paper, we first use a state-of-the-art Topic-to-Essay generation model to generate texts conditional on some topic words. Then, by analyzing the generated text, we propose an annotation framework, and then quantify the main drawbacks of current NLG, including poor semantic coherence, content duplication, logic errors, and repetition. It shows that the text generated by the current sequence-to-sequence model is still far from human expectation.
• Co-author:Xingyuan Chen,Peng Jin
• First Author:Cai Ping
• Indexed by:Academic papers
• Correspondence Author:Hongjun Wang
• Document Code:20211910316120
• Discipline:Engineering
• First-Level Discipline:Computer Science and Technology
• Translation or Not:no
• Date of Publication:2021-05-27