On benchmarking of document analysis systems

Thes paper is a contribution to the development of evaluation methods in general and in particular of the evaluation of document analysis systems (DAS). We give a general concept of benchmarking DAS and present a method to benchmark single modules of a DAS where we focus on the definition and generation of ground truth (GT), especially of the image processing modules of a DAS. The proposed method describes how to build a database with specific ground truth for the output of each module of interest. Another method to build a database for benchmarking more generally is to use synthetic data