ارايه يك سامانه هوشمند و معناگرا براي ارزيابي سامانه‌هاي خلاصه‌ساز متون

Proposing an intelligent and semantic-based system for Evaluating Text Summarizers

امروزه با افزايش منابع متني در شبكه جهاني وب، هر روز بر گستره اطلاعات قابل دسترس براي كاربران افزوده مي‌شود؛ بنابراين جهت نگهداري و بازيابي و پردازش آنها از سامانه‌هاي خلاصه سازي خودكار متن، استفاده مي كنيم. ميزان كيفيت خلاصه سازهاي ماشيني، توسط انسان ها مورد بررسي قرار مي گيرد؛ اما اين كار نيروي متخصص و زمان زيادي را مي طلبد و هزينه‌بر خواهد بود؛ بنابراين براي حل اين مشكل، در اين مقاله سامانه‌اي به نام TabEval براي ارزيابي سامانه‌هاي خلاصه سازي خودكار متن ارايه شده است. اين سامانه با ايده و معماري جديد به محاسبه ميزان تشابه ظاهري و معنايي بين خلاصه سامانه‌اي و خلاصه هاي انساني (خلاصه هاي ايده آل) مي پردازد. براي محاسبه ميزان تشابه معنايي از شبكه واژه ها استفاده مي‌شود. خروجي حاصل از اين سامانه توسط نيروهاي متخصص در زمينه ادبيات زبان فارسي مورد بررسي قرار گرفت. نتايج حاصل از بررسي ها حاكي از اين بود كه اين سامانه همانند انسان، هوشمندانه عمل مي كند.

Nowadays summarizers and machine translators have attracted much attention to themselves, and many activities on making such tools have been done around the world. For Farsi like the other languages there have been efforts in this field. So evaluating such tools has a great importance. Human evaluations of machine summarization are extensive but expensive. Human evaluations can take months to finish and involve human labor that cannot be reused. In this paper, we propose a method of automatic machine summarization evaluation that is quick, inexpensive, and language-independent, that correlates highly with human evaluation, and that has little marginal cost per run. This method has the metrics of determining auto summaries’ quality, through comparing them to the summaries produced by Human (ideal summaries). These metrics measures overlapping of system summaries and human ones in number of units like n-tuples, words string and pairs of words. Certainly for semantic comparing of texts in case of review summaries, the appearance of words are not enough and using of WordNet seems to be necessary. In the proposed method words network is used with an appropriate idea and has improved evaluation results significantly. The proposed method is the first method for the Persian language. Performance measurement of the tool was done during a specified and standard procedure and the results indicate acceptable yield of it. We present this method as an automated understudy to skilled human judges which substitutes for them when there is need for quick or frequent evaluations.