Use-cases for performance evaluation of human-mind inspired control units

With the advent of artificial general intelligence architectures, comparison between different architectures became more complicated. Small isolated tasks like box stacking, path finding, chess, or collecting objects is sufficient for highly specialized algorithms like the ones used in artificial intelligence. The holistic control systems are designed to cope with rich, dynamic, and complex real world environments and tasks. The approaches used to design these architectures differ in many ways but all share the goal to create human like “intelligence.” Each project uses its own approach to evaluate its success which is usually tightly coupled with the target application. We propose the usage of a simulated virtual world with embodied autonomous agents. These agents should be equipped with control systems from the architectures that should be compared and exposed to situations where their performance is assessed. The situations-or test-cases-have to be designed with tests from psychology in mind. Thus in a top-down design approach, tests from psychology have to be refined and converted such that they can be implemented into the simulation. We discuss different approaches and introduce the use-cases developed for project ARS.