RCSUM: To build a summarization system directly generating summaries with evaluation metrics

Several automated metrics have been adopted by document understanding conference (DUC) in recent years as they offered great advantages in the area of summarization. However, there´re still no evaluation metrics which can be directly used to generate Summaries, mainly because that human reference summaries are indispensable for these metrics. Here we report that our group first discovered and developed RCSUM, a summarization system that can generate summaries with evaluation metrics. RCSUM was developed from ROUGE-C, a new fully automated evaluation metrics recently developed in our group, which can use ROUGE in an absolutely manual-independent way, and reserved capabilities of distinguishing good or bad summaries. Experiments conducted on the 2001 to 2005 DUC data showed that, evaluated by ROUGE-2 as well as ROUGE-SU4, our system performed very well.