Multi-Agent based automatic evaluation system for classification algorithms

The current researches on classification usually focus on proposing novel algorithms and improving existent ones with better performances. However, although a classification algorithm is able to perform well for some given data sets, does it mean to any other data sets? And given several algorithm candidates, which one is the best for your classification problem? A practical solution to the above questions is to evaluate possible situations, which is extremely time-consuming and resource-consuming. In this paper we propose a distributed computing environment based on Multi-Agent technology to facilitate this evaluation process. In this computing environment we compare the performances of the same algorithm on different data sets, and different algorithms on the same data set. Experiments show that autonomic agents can run simultaneously and automatically on different computing hosts to achieve high availability, and this scheme can save the total evaluation time greatly. Therefore, this scheme will help us easily select the proper algorithm for a given classification problem according to different evaluation measures.